SDKsSDK features

Customizing AI Configs

AI SDKs are in an alpha version

The AI SDKs are designed for use with the AI Configs product and are currently in alpha versions.

Overview

This topic explains how to customize an AI Config. This feature is available for AI SDKs only.

About AI Configs

The AI Config feature customizes an AI Config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime.

This feature requires an AI Config key, the context that encounters the AI Config in your application, a fallback value, and, optionally, additional variables used in the prompt template.

This feature returns the value of the customized AI Config variation to use for this context, based on the AI Config’s targeting rules. This variation includes the model and a customized messages for the context. Then, you can use the value of the AI Config variation in your AI model generation. You need to use the AI Config feature each time you generate content from your AI model.

Customize messages

Customizing the AI Config means that LaunchDarkly substitutes context attributes and optional variables into the messages associated with the AI Config variation. If the message has multiple messages, they are all customized and returned. If the variation cannot be retrieved, for example because the AI Config key does not exist, or because the SDK cannot connect to LaunchDarkly, then the SDK returns the fallback value.

Syntax for customization

When you create each message in an AI Config variation, use the following syntax if you want to substitute context attributes or other variables in the message:

  • Use two curly braces to indicate variables in your application code. For example:

    • Enter This is an {{ example }} message in the message in your AI Config variation in the LaunchDarkly UI. The variable name cannot have a period (.) in it.
    • In the SDK, pass a dictionary or key-value pair with example and your example_value to the config call.

  • Use two curly braces, the ldctx prefix, and dot (.) notation to indicate context attributes. For example:

    • Enter Describe the typical weather in {{ ldctx.city }} if you’d like to replace {{ ldctx.city }} with the city attribute from each context that encounters this AI Config. The context attribute cannot have a period (.) in it.
      • If your context attribute has multiple fields within it, you can use . as a delimiter in your AI Config message. For instance, if the context has an “address” attribute that includes “city” and other fields, then you could use {{ldctx.address.city}} in your prompt, and the value of the “city” field will be substituted when you customize the AI Config.
    • In the SDK, the context is a required parameter to the config call. You do not need to pass any additional information to the config call.

Customization is based on context

The AI Config feature adds a context to the Contexts list, if a context with the same key does not already exist. However, each SDK customizes the AI Config based only on the object you provide in the config call. In other words, the SDK does not automatically use the attributes shown on the Contexts list, and attributes are not synchronized across SDK instances. You must provide all relevant attributes for each customization in order for variables in your prompt to be substituted correctly. To learn more, read Context configuration.

AI SDKs

This feature is available for all of the AI SDKs:

.NET AI

The Config function customizes the AI Config. You need to call Config each time you generate content from your AI model.

This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI Config variation have their values set to the context attributes and variables you pass to Config. Then, you can pass the customized prompt directly to your AI.

Config calls take the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use LdAiConfig.Disabled as a fallback value, and then check for this during your application’s error-handling. Alternatively, you can create a custom LdAiConfig object using LdAiConfig.New().

Here is an example of calling the Config method:

.NET AI SDK
1var fallbackConfig = LdAiConfig.New()
2  .SetModelName("my-default-model")
3  .SetModelParam("temperature", LdValue.Of(0.8))
4  .AddMessage("", Role.system)
5  .SetModelProviderName("my-default-provider")
6  .SetEnabled(true)
7  .Build()
8
9var tracker = aiClient.Config(
10  "ai-config-key-123abc",
11  context,
12  fallbackConfig,
13  new Dictionary<string, object> {
14    { "exampleCustomVariable", "exampleCustomValue" }
15  }
16);

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read Config.

Go AI

The Config function customizes the AI Config. This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI Config variation have their values set to the context attributes and variables you pass to Config. Then, you can pass the customized prompt directly to your AI.

Config calls take the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty, disabled Config as a fallback value, by calling ldai.Disabled(), and then check for this during your application’s error-handling. Alternatively, you can create a custom Config object using NewConfig.

Here is an example of calling the Config method:

Go AI SDK
1fallbackValue := NewConfig().
2  Enable().
3  WithModelName("my-default-model").
4  WithModelParam("temperature", ldvalue.Float64(0.8)).
5  WithMessage("", datamodel.System).
6  WithProviderName("my-default-provider").
7  Build()
8
9cfg, tracker := aiClient.Config("ai-config-key-123abc", context, fallbackValue, map[string]interface{}{"exampleCustomVariable": "exampleCustomValue"})

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read Config.

Node.js (server-side) AI

The config function customizes the AI Config. You need to call config each time you generate content from your AI model.

This function returns the customized prompt and model. Customization means that any variables you include in the prompt messages when you define the AI Config variation have their values set to the context attributes and variables you pass to config. Then, you can pass the customized prompt directly to your AI.

config calls take the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application’s error-handling. Alternatively, you can construct a JSON object, for example with values from one of the AI Config variations you have created in the LaunchDarkly UI.

Here is an example of calling the config method:

Node.js (server-side) AI SDK (TypeScript)
1const fallbackConfig = {
2  model: {
3    name: 'my-default-model',
4    parameters: { temperature: 0.8 }
5  },
6  messages: [ { role: 'system', content: '' } ],
7  provider: { name: 'my-default-provider' },
8  enabled: true,
9};
10
11const aiConfig: LDAIConfig = aiClient.config(
12  'ai-config-key-123abc',
13  context,
14  fallbackConfig,
15  { 'exampleCustomVariable': 'exampleCustomValue' },
16);

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read config.

Python AI

The config function customizes the AI Config. You need to call config each time you generate content from your AI model.

This function returns the customized prompt and model along with a tracker instance for recording prompt metrics. Customization means that any variables you include in the prompt messages when you define the AI Config variation have their values set to the context attributes and variables you pass to config. Then, you can pass the customized prompt directly to your AI.

config calls take the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application’s error handling. Alternatively, you can construct a JSON object, for example with values from one of the AI Config variations you have created in the LaunchDarkly UI.

Here is an example of calling the config method:

Python AI SDK
1key = 'ai-config-key-123abc'
2context = Context.builder('context-key-123abc') \
3  .kind('user') \
4  .set('name', 'Sandy') \
5  .build()
6fallback_value = AIConfig(
7  enabled=True,
8  model=ModelConfig(
9    name="my-default-model",
10    parameters={"temperature": 0.8},
11  ),
12  messages=[LDMessage(role="system", content="")],
13  provider=Provider(name="my-default-provider"),
14)
15variables = { 'example_custom_variable': 'example_custom_value' }
16
17config, tracker = aiclient.config(key, context, fallback_value, variables)

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read config.

Ruby AI

The config function customizes the AI Config. You need to call config each time you generate content from your AI model.

This function returns the customized prompt and model, along with a tracker instance for recording prompt metrics. Customization means that any variables you include in the prompt messages when you define the AI Config variation have their values set to the context attributes and variables you pass to config. Then, you can pass the customized prompt directly to your AI.

config calls take the AI Config key, a Context, a fallback value, and optional additional variables to substitute into your prompt.

The fallback value is the value of the AI Config variation that your application should use in the event of an error, for example, if the AI Config key is not valid, or if there is a problem connecting to LaunchDarkly. You can use an empty JSON object as a fallback value, and then check for this during your application’s error handling. Alternatively, you can construct a JSON object, for example with values from one of the AI Config variations you have created in the LaunchDarkly UI.

Here is an example of calling the config method:

Ruby AI SDK
1key = 'ai-config-key-123abc'
2context = LaunchDarkly::LDContext.create({ key: 'example-user-key', kind: 'user', name: 'Sandy' })
3fallback_value = LaunchDarkly::AI::AIConfig.new(
4    enabled: true,
5    model: LaunchDarkly::AI::ModelConfig.new(
6        name: 'my-default-model',
7        parameters: { 'temperature': 0.8 }
8    ),
9    messages: [{ role: 'system', content: '' }],
10    provider: { name: 'my-default-provider' }
11)
12variables = { 'example_custom_variable' => 'example_custom_value' }
13
14ai_config = ai_client.config(key, context, fallback_value, variables)

After the function call, you can view the context that you provided to it on the Contexts list.

To learn more, read config.