AI Configs
Overview
The topics in this category explain how to use LaunchDarkly to manage your AI Configs. You can use AI Configs to customize, test, and roll out new large language models (LLMs) within your generative AI applications.
An AI Config is a single resource that you create in LaunchDarkly to control how your application uses large language models. It lets teams manage model configuration outside of application code, enabling safer iteration, experimentation, and releases without redeploying.
AI Configs support two configuration modes:
- Completion mode: Use messages and roles to configure prompts for single-step completions. To learn more, read Create and manage AI Config variations.
- Agent mode: Use instructions and optional tools to configure multi-step workflows. To learn more, read Agents in AI Configs.
Agent mode does not create a separate resource. AI Configs in both modes use the same underlying concepts for variations, targeting, monitoring, experimentation, and lifecycle management.
With AI Configs, you can:
- Manage your model configuration outside of your application code. This means you can update model details and messages at runtime, without deploying changes. Teammates who have LaunchDarkly access but no codebase familiarity can collaborate and iterate on messages.
- Upgrade to new model versions as soon as they are available and roll out changes gradually and safely.
- Add new model providers and progressively shift production traffic between them.
- Compare variations to determine which performs better based on cost, latency, satisfaction, or other metrics.
- Run experiments to measure the impact of generative AI features on end user behavior.
AI Configs also support advanced use cases such as retrieval-augmented generation and evaluation in production. You can:
- Track which knowledge base or vector index is active for a given model or audience.
- Experiment with different chunking strategies, retrieval sources, or prompt and instruction structures.
- Evaluate outputs using side-by-side comparisons or AI Config-as-a-judge patterns implemented in your application.
- Build guardrails into runtime configuration using targeting rules to block risky generations or switch to fallback behavior.
- Apply different safety filters by user type, geography, or application context.
- Use live metrics for satisfaction, factuality, and hallucination detection to guide rollouts.
These capabilities let you evaluate model behavior in production, run targeted experiments, and adopt new models safely without being locked into a single provider or manual workflow.
Availability
AI Configs is an add-on feature. Access depends on your organization’s LaunchDarkly plan. If AI Configs does not appear in your project, your organization may not have access to it.
To enable AI Configs for your organization, contact your LaunchDarkly account team. They can confirm eligibility and assist with activation.
For information about pricing, visit the LaunchDarkly pricing page or contact your LaunchDarkly account team.
How AI Configs work
Every AI Config contains one or more variations. Each variation includes a model configuration and, optionally, one or more messages. You can also define targeting rules, just like you do with feature flags, to make sure that particular messages and model configurations are served to particular end users of your application.
Then, within your application, you use one of LaunchDarkly’s AI SDKs. The SDK determines which messages and model your application should serve to which contexts. The SDK can also customize the messages based on context attributes and other variables that you provide. This means both the messages and the model evaluation are modified to be specific to each end user at runtime. You can update your messages, specific to each end user, without redeploying your application.
After you use this customized config in your AI model generation, you can use the SDK to record various metrics, including generation count, tokens, and satisfaction rate. These appear in the LaunchDarkly user interface for each AI Config variation.
The topics in this category explain how to create AI Configs and variations, update targeting rules, monitor related metrics, and incorporate AI Configs in your application.
Additional resources
In this section:
- Quickstart for AI Configs
- Create AI Configs and variations
- Target with AI Configs
- Monitor AI Configs
- Manage AI Configs
- Run experiments with AI Configs
- AI Configs and information privacy
- Agents in AI Configs
In our guides:
In our SDK documentation:
