What is Pathways Language Model (PaLM)?

The Pathways Language Model is the name of a family of AI large language models developed by Google. The effort gets its name from a Google Research initiative to create what researchers dubbed pathways, in an approach designed to build a single powerful model that could serve as a foundation for multiple use cases.

There are multiple versions of the Pathways Language Model (PaLM). Among the versions of PaLM 2 are Med-PaLM 2, which is fine-tuned for life sciences and medical information; and Sec-PaLM, which is focused for use in cybersecurity deployments to expedite threat analysis.

In May 2023, Google publicly stated that its Bard conversational AI technology is powered by PaLM 2. The PaLM 2 large language model (LLM) also enables generative AI capabilities in the Google Workspace suite of applications -- including Gmail and Docs -- as well as Google Cloud with a technology known as Duet AI.

PaLM -- and more specifically PaLM 2 -- can provide many functions, including the following:

PaLM uses a transformer neural network-based model commonly referred to as a transformer. At a basic level, PaLM is similar to rival transformer-based models, including OpenAI's GPT-3 and GPT-4 models.

PaLM uses the Google-developed Pathways machine learning system to train a model across multiple pods of tensor processing units. The model uses a technique known as few-shot learning that lets it learn from a limited number of labeled examples -- or shots -- to help it quickly adapt and generalize new tasks or classes with minimal data labeling.

As a transformer network, PaLM understands and creates patterns across content, including text and code. The transformer model undergoes a comprehensive learning process, uncovering statistical patterns and connections that exist among words and phrases within content. This acquired knowledge empowers PaLM to generate responses that are both coherent and relevant in various contexts.

While PaLM is powerful, it has the following limitations on use and capabilities, as well as other items of concern.

"Prompted dialog systems built from PaLM 2 continue to produce toxic language harms, and to exhibit patterns of bias in how those harms vary by language and queries related to identity terms," Google researchers wrote in the "PaLM 2 Technical Report."

There are many similarities and differences between PaLM and OpenAI's GPT-3 -- and the more recent GPT-4 -- LLMs. Both sets of technologies are generally referred to as generative AI, and they both benefit from the use of a transformer model for deep learning. Both technologies can also create, summarize and understand text.

Developer

Google DeepMind

OpenAI

Chatbot interface

Bard

ChatGPT

Code generation

Fully integrated model

Draws on data from OpenAI's Codex LLM

Multilingual capabilities

PaLM 2 currently supports more than 40 languages

GPT-4 currently supports 26 languages

Google announced PaLM in April 2022, with the initial version of the language model providing 540 billion parameters.

Google researchers provided early metrics on PaLM performance in a research paper titled "PaLM: Scaling Language Modeling with Pathways," including full details explaining the innovations the model introduces. The model remained private until March 17, 2023, when Google provided an initial set of public APIs to let developers try the model. On May 10, 2023, PaLM 2 was publicly announced at the Google I/O conference.

At the time of the PaLM 2 launch, Google didn't publicly disclose the size of the model in terms of parameters, though it made many claims about the model being larger, more capable and overall providing better performance than its initial model.

Google did disclose that PaLM 2 is being used as the foundational AI behind many of its generative AI efforts, including Bard.

Looking forward, it's unclear if there will be a PaLM 3 model. At the Google I/O 2023 event, company executives said the next-generation LLM being developed by the organization is called Gemini. It's not clear what -- if any -- relationship Gemini has to the PaLM-based approach.

Apple iOS is a proprietary mobile operating system that runs on mobile devices such as the iPhone and iPad.