Teams building with AI face pressure to both move quickly and avoid risk. Every change to a model or prompt can help performance, but it can also cause unexpected behavior. As more teams bring AI tools into their release processes, the cost of guessing in pre-prod rises. LaunchDarkly and AWS can help you iterate quickly while maintaining safe releases.
With AI Configs and Amazon Bedrock, you can manage and test models, prompts, and parameters in real time. Bedrock Guardrails help protect against unsafe or off-policy responses. LaunchDarkly progressive rollouts and instant rollbacks let you measure performance and refine code without having to redeploy.
Managing AI behavior with LaunchDarkly AI Configs
AI Configs is a LaunchDarkly resource for managing models, prompts, parameters, and tools at runtime. It gives teams a control plane for how AI behaves in production.
Instead of hard-coding model details into your application, you define them in LaunchDarkly and stream updates through the SDK. You can adjust parameters such as model selection, prompt content, or temperature (the setting that controls how deterministic or creative model outputs are) without redeploying. Every change is versioned, reviewed, and can be rolled back instantly.
On AWS, AI Configs connect directly to Amazon Bedrock. You can point a configuration to any Bedrock model, such as Anthropic Claude or Amazon Nova, and compare multiple versions in real time. This makes it easy to evaluate cost, latency, and accuracy across models or prompts while your application stays live.
With AI Configs, teams can:
- Iterate on models and prompts in production
- Control rollout and targeting by user segment
- Measure performance and roll back automatically if metrics drop
AI Configs make experimentation and safety part of the same workflow.
Testing model performance in real time with Amazon Bedrock
AI Configs make it easy to compare multiple models and prompts in production. Using Amazon Bedrock, you can evaluate performance, cost, and quality across model providers with live user traffic instead of offline tests.
We used a financial services chatbot to test how different Amazon Bedrock models performed in production. The goal was to identify which model provided the best balance of accuracy and efficiency before scaling it to all users.
Using LaunchDarkly AI Configs, modes like Anthropic Claude Sonnet 4, Claude 3.5, Haiku 3.5, and Amazon Nova Pro were defined as variations of the same configuration. This allowed the team to:
- Deliver multiple models to real users under controlled conditions
- Measure metrics such as cost, latency, and CSAT in real time
- Roll back instantly if a variation underperformed
Progressive rollouts added another layer of control. Instead of switching all users to a new model at once, we gradually exposed the change to specific segments and monitored live results. When the data confirmed that Nova Pro performed on par with larger models at a lower cost, we promoted it to production with a single click; no redeploy required.
This approach gives teams an evidence-based way to evolve their AI systems. They can adopt new models faster, validate impact in real environments, and help ensure every update improves both performance and reliability.
Building safer AI with Bedrock guardrails
Guardrails protect user interactions by inspecting both the prompt and the model response. They look for sensitive content, policy violations, and prompt attacks. Typical checks include PII, hateful or violent speech, and injection attempts.
In our setup, every conversation runs through Bedrock Guardrails. When a policy is triggered, the application receives a clear signal to block or mask the response. LaunchDarkly listens for that signal and immediately:
- Disables the active AI Config for the session
- Routes the user to a safe fallback, such as a live agent or a non-AI workflow
- Logs context and metrics for review
- Optionally requires approval before re-enabling the config
This creates a circuit breaker for AI in production. A bad response is stopped in real time, and the system rolls back to a safer state without a redeploy. You keep the conversation stable while you investigate, adjust prompts or parameters, and promote a fixed version when ready.
Streamlining AI development with Amazon Q and LaunchDarkly MCP
Beyond managing AI in production, teams can also use generative AI to build and maintain their AI configurations. Amazon Q, the AWS generative AI assistant for developers, can connect directly to LaunchDarkly through the Model Context Protocol (MCP). This allows developers to inspect, modify, and create AI Configs using natural language inside their development environment.
For example, a developer used Amazon Q to review the application code for a chat interface built with Streamlit (an open-source Python framework for building simple web apps) and to understand how LaunchDarkly AI Configs were defined. Q analyzed the source files, retrieved the live AI Config from LaunchDarkly, and summarized key details, including model variations, system prompts, and user context.
The developer then asked Q to propose a prompt improvement. After reviewing the suggestion, they approve it, and Q updates the configuration in LaunchDarkly. No manual edits or redeployments are required. Human review stays in the loop, so changes still follow approvals and audits.
This integration shortens the feedback loop for AI teams. Developers can work faster, product managers can safely propose changes, and every update remains visible and reversible in LaunchDarkly.
Get started with LaunchDarkly and AWS
Teams like Poka, a connected worker platform, are already using LaunchDarkly and Amazon Bedrock to bring new models into production. By treating models and prompts as configurations, Poka was able to evaluate new Bedrock models on launch day and update production systems in minutes rather than days.
You can do the same. LaunchDarkly and AWS give you a shared control plane for building, testing, and improving AI applications with confidence. Sign up for a product demo.
Like what you read?
Get a demo
