Create and manage AI model configurations

Overview

This topic explains how to create and manage custom models to use in AI Configs.

An AI Config is a resource that you create in LaunchDarkly. You can use AI Configs to customize, test, and roll out new large language models (LLMs) within your generative AI applications. Within each AI Config, you define one or more variations, each of which includes a model configuration, and one or more messages.

When you create a variation, you have the option to use a standard model, also called a “global” model, from the list provided by LaunchDarkly.

You can also add your own model and set up its parameters. This is called an AI model configuration. After you create an AI model configuration, you can use it in any variation for any AI Config in your project.

You can set both global models and custom models as restricted, which means that no one in your organization can use those model configurations in any AI Config variations. This lets you effectively limit the models that are available in each LaunchDarkly project.

This topic covers the following topics:

• Create an AI model configuration
• View AI model configurations
• Restrict an AI model configuration
• Delete an AI model configuration

Create an AI model configuration

You can create an AI model configuration either while you are creating an AI Config variation, or from the AI model configs tab under Project settings.

Create an AI model configuration from a variation

To create an AI model configuration from an AI Config variation:

1. Navigate to the Variations tab of the AI Config and select a variation.
2. Open the Select a model menu.
3. Choose + Add a model.
4. Follow the steps for completing the “Add custom model” dialog, below.

Create an AI model configuration from project settings

To create an AI model configuration from your Project settings page:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.
4. Click Add AI model config.
5. Follow the steps for completing the “Add custom model” dialog, below.

Complete the “Add custom model” dialog

To complete the “Add custom model” dialog:

1. Enter a Name for your AI model configuration.

• The name appears in the Select a model dropdown when you create a variation for an AI Config.
• You also need the name if you want to create a custom role that allows or denies access to this AI model configuration.

2. Select a Provider.
3. Enter a Model ID.

• The model ID is the identifier for the model that the LLM provider uses. This must match the identifier from the provider exactly.
• You also need the model ID if you want to create a custom role that allows or denies access to this AI model configuration.

4. (Optional) Click + Add suggested parameters. The menu displays model parameters, their types, and their default values.

• Select a model parameter from the list.
• Set its value in the dialog. This will be the default value when you choose this model in an AI Config variation. You can set a different value in the AI Config variation if needed.
• When you customize an AI Config in your SDK, you can access these parameters in the model section of the LDAIConfig.

5. (Optional) Click + Add custom parameters.

• Enter the additional parameters you want to include. Your entry must be a valid JSON object.
• These parameters and the values you specify here will always be used when you choose this model in an AI Config variation.
• When you customize an AI Config in your SDK, you can access these parameters in the model.custom section of the LDAIConfig.

6. Click Save.

Here is an example:

View AI model configurations

To view all of the AI model configurations in a project:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.

You can filter the list by model type and availability:

You can use the search box to find models by entering all or part of the model name or ID.

Restrict an AI model configuration

You can set any AI model configuration as restricted, which means that no one in your organization can use those models in any AI Config variations. This lets you effectively limit the models that are available in each LaunchDarkly project.

To restrict an AI model configuration:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.
4. Click the three-dot overflow menu and select Restrict.

To make an AI model configuration available again, click the three-dot overflow menu and select Make available.

Delete an AI model configuration

To delete an AI model configuration:

1. Click the project dropdown. The project menu appears.
2. Select Project settings.
3. Select AI model configs. The AI model configs list appears.
4. Click the three-dot overflow menu and select Delete.

• Alternatively, click the name of the AI model config you want to delete. Then click Delete AI model config from the detail page.

You can only delete custom AI model configurations. You cannot delete the global AI model configs provided by LaunchDarkly.

You cannot delete an AI model configuration if it is currently used in a variation for an AI Config.