Supported Models

Understand supported large language models (LLMs) from multiple providers for embedded features, such as Prompt Builder, and Models API. Identify the Salesforce-managed models that are available out of the box. Learn how you can bring your own model (BYOLLM) by using Einstein Studio.

All Agentforce reasoning engine calls use OpenAI GPT-4o, and in some cases Azure OpenAI GPT-4o. You can't select a different model for AI agents. However, custom actions that execute prompt templates or Apex or call the Models API can use any Salesforce-managed or BYO model.

This table lists the API names for all the standard configuration models in Einstein Studio. In addition to these models, you can use the API name from any custom model configuration in Einstein Studio.

To see details, such as model version and supported regions, see Large Language Model Support in Salesforce Help.

* Salesforce Trust Boundary: Anthropic models on Amazon are operated on Amazon Bedrock infrastructure entirely within the Salesforce Trust Boundary. In contrast, other models are operated by Salesforce partners, either inside a shared trust zone or through the LLM provider directly using Einstein Studio’s bring your own LLM (BYOLLM) feature.

The Models API supports Einstein Studio’s bring your own LLM (BYOLLM) feature, which currently supports Amazon Bedrock, Azure OpenAI, OpenAI, and Vertex AI from Google as foundation model providers. With BYOLLM, you can add a foundation model from a supported provider, configure your own instance of the model, and connect to the model using your own credentials. Although inference is handled by the LLM provider, the request is still routed through the Models API and Trust Layer features are fully supported.

Using a BYOLLM model with the Models API is the same as any other model. Look up the API Name of the configured model in Einstein Studio and use it as the {modelName} in the REST endpoint path or as the modelName property of the Apex request object.

To see the list of foundation models that you can add in Einstein Studio with BYOLLM, see Large Language Model Support in Salesforce Help.

To learn more about BYOLLM, see Bring Your Own Large Language Model in Einstein 1 Studio on the Salesforce Developers Blog.

The Bring Your Own Large Language Model (BYOLLM) Open Connector is designed to provide powerful AI solutions to customers, independent software vendors (ISVs), and internal Salesforce teams. With this connector, you can connect the Einstein AI Platform to any language model, including custom-built models.

The BYOLLM Open Connector is a commitment to community-driven growth and innovation. By allowing users to integrate any LLM—from those models hosted on major cloud platforms to those models developed in-house—we're opening up a world of possibilities for enhanced, bespoke AI applications. This capability not only caters to the needs of large enterprises looking to leverage specific models like IBM Granite or Databricks DBRX, but also supports smaller teams eager to experiment with open-source models. With features designed to ensure ease of use, such as a streamlined UX in Einstein Studio and API specifications closely based on the OpenAI API, this connector empowers our users to enhance their AI-driven applications while maintaining high standards of security and compatibility.

See the Einstein AI Platform GitHub repository for API specifications and example code for the LLM Open Connector.

To choose the right model for your application, consider these criteria.

Capabilities: What can the model do? Advanced models can perform a wider variety of tasks (usually at the expense of higher costs and slower speeds—or both). The ability to follow complex instructions is a key indicator of model capabilities.

Cost: How much does the model cost to use? For details on usage and billing, see Einstein Usage.

Quality: How well does the model respond? The quality of model responses can be hard to measure quantitatively, but a good place to start is the LMSYS Chatbot Arena.

Speed: How long does it take the model to complete a task? Includes measures of latency and throughput.

For benchmarks and evaluations of LLMs and embedding models, see these resources.

• Artificial Analysis: Aggregated data on LLM performance.
• LLM Benchmark for CRM: Evaluation of LLMs for Sales and Service use cases. Provided by Salesforce AI Research.
• LMSYS Chatbot Arena: Human scoring of LLMs. Anyone can participate!
• MTEB Leaderboard: Benchmarks for embedding models from Huggingface.
• SEAL Leaderboard: Evaluations of LLMs using private datasets from Scale AI.

The context window determines how many input and output tokens the model can process in a single request. The context window includes system messages, prompts, and responses.

All models are currently limited to a context size of 65,536 tokens when data masking is turned on in the Einstein Trust Layer. To turn off data masking and use the full context window, see Set Up Einstein Trust Layer in Salesforce Help.

For more information about the context window for individual models, see the model provider site.

• Amazon Bedrock: Anthropic Claude
• Azure OpenAI
• OpenAI
• Vertex AI (Google)

A geo-aware model minimizes latency by automatically routing your LLM request to a nearby data center based on where Data Cloud is provisioned for your org.

Proximity to the nearest LLM server is determined by the region in which your Einstein generative AI platform instance is located. If you enabled the Einstein generative AI platform on or after June 13, 2024, then your Einstein generative AI platform region is the same as your Data Cloud region (Data Cloud: Data Center Locations). Otherwise, contact your Salesforce account executive to learn where it’s provisioned.

To learn more about geo-aware routing, see Geo-Aware LLM Request Routing in Salesforce Help.

Announcements for new models and model deprecations are part of the Einstein Platform release notes on a monthly basis.

Model deprecation is the process where a model provider gradually phases out a model (usually in favor of a new and improved model). The process starts with an announcement outlining when the model will no longer be accessible or supported. The deprecation announcement usually contains a specific shutdown date. Deprecated models are still available to use until the shutdown date.

After the shutdown date, you won’t be able to use that model in your application and requests to that model will be rerouted to a replacement model. We recommend that you start migrating your application away from a model as soon as its deprecation is announced. During migration, update and test each part of your application with the replacement model that we recommend. For more details about deprecated models, see Large Language Model Support in Salesforce Help.

• Model API Names
• Access Models API with Apex
• Access Models API with REST
• Rate Limits for Models API
• Models REST API Reference