For clean Markdown of any page, append .md to the page URL. For a complete documentation index, see https://launchdarkly.com/docs/llms.txt. For full documentation content, see https://launchdarkly.com/docs/llms-full.txt. This file is very large and may time out. For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://launchdarkly.com/docs/mcp.

​

LaunchDarkly AgentControl — Agent Onboarding Prompt

Keys supplied with this prompt. Check this first. The message that pointed you at this page may already carry the developer’s credentials, for example use LAUNCHDARKLY_SDK_KEY=sdk-... and LAUNCHDARKLY_AI_CONFIG_KEY=my-config. The LaunchDarkly in-app onboarding sends them that way. When those values are present:

• Treat them as the credentials for this setup. Do not fetch the SDK key with get-project, and do not send the developer to Account settings → Environments to look one up.
• Skip creating a config in Phase 2 Step 2.5. A config key in the prompt means the config already exists.
• Still run the secret-handling question in Phase 2 Step 3 before writing either value to .env or anywhere else. Having the values does not mean you may choose where they land.
• Do not echo the full SDK key back in chat.

Naming note for the agent. AgentControl is the LaunchDarkly product for managing configs (model + prompt + parameters + tools, served from LaunchDarkly at runtime to drive your AI features). The product was previously called AI Configs; the user-facing terminology is now AgentControl (product) and configs (the things you create inside it). Many technical identifiers still use the old ai-config / ai_config / AI prefix and have not been renamed. This is intentional. The SDK packages, classes, MCP tool names, environment variable names, and documentation URLs ship under the legacy names, and changing them in code without an SDK release would break the user’s app. Keep using:

• Legacy AI SDK package names: launchdarkly-server-sdk-ai, @launchdarkly/server-sdk-ai, @launchdarkly/server-sdk-ai-openai
• Legacy AI SDK classes / functions: LDAIClient, AICompletionConfigDefault, AIAgentConfigDefault, completion_config(), agent_config(), create_model(), create_judge()
• MCP tool names: setup-ai-config, create-ai-config-variation, update-ai-config-variation, create-ai-tool, list-ai-configs, get-ai-config, update-ai-config-rollout, etc.
• Environment variables: LAUNCHDARKLY_AI_CONFIG_KEY, LAUNCHDARKLY_AI_JUDGE_KEY
• Documentation URLs: https://docs.launchdarkly.com/home/ai-configs/..., https://docs.launchdarkly.com/sdk/ai/...

Use AgentControl and config / configs in prose, headings, status messages, and anything you say to the user. Use the legacy ai-config identifiers in code, MCP calls, env vars, and URLs. If the user asks why, explain that the rename is rolling out across the product surface and the identifiers update on their normal release cadence. The current Python and Node.js AI SDKs are the exception to the list above. They ship under new package names (launchdarkly-ai-server, @launchdarkly/ai-node, and a handler package per provider) and a new function surface (config(), invoke(), graph()), and they read the SDK key from LD_SDK_KEY rather than LAUNCHDARKLY_SDK_KEY. The packages listed above are the legacy AI SDKs, which are now in maintenance mode. New applications should start on the current AI SDKs. Read Which AI SDK to use before you install anything.

​

Core principles

• Detect before asking — infer what you can from the codebase; ask only when ambiguous
• Inspect before mutating — understand the codebase before changing anything
• Do not change business logic — the LaunchDarkly integration is purely additive
• Wrap, don’t replace — keep existing agent code intact; wrap it with the LaunchDarkly AI SDK to pull model config and instructions from LaunchDarkly at runtime
• Follow existing code style and project conventions
• Keep output concise — do not generate extra documentation or summary files
• Ask before changing non-LaunchDarkly dependencies — installing the LaunchDarkly AI SDK packages named in this prompt is in-scope. Anything else — upgrading existing packages to resolve peer-dependency warnings, downgrading the user’s framework version, running npm audit fix, bumping react/node/etc. — requires explicit user approval before you run the command or edit the manifest. If the install reports peer conflicts, surface the exact error, propose the minimal change, and wait for the user to confirm before proceeding. The user’s existing dependency versions may be pinned for reasons you cannot see (downstream apps, internal compatibility constraints, governance policies); silently bumping them is a high-cost mistake even when it makes the build pass.
• Treat SDK keys and provider keys as last resort — never fetch, write, or paste real keys without explicit user consent. When keys came in with the prompt, use them rather than fetching more, but the consent question still governs where they get written. The structured consent question in Phase 2 Step 3 is mandatory before writing anything to .env or any other secret store. Some users keep .env under tight controls (CI-only, secrets manager, encrypted vault) and an agent silently dropping a key into it is a security incident, not a convenience.
• Prefer MCP over the UI when MCP can do the job — when the LaunchDarkly MCP servers are connected, use them for any operation they support so the user stays inside the agent context instead of bouncing to the UI. Sending the user to the UI for something the agent could have done in one MCP call is a worse experience and a missed opportunity to demonstrate the platform. Discover available tools dynamically: at the start of any LaunchDarkly operation, list the tools exposed by both MCP servers and treat that live list as the source of truth — the MCP capability map below is a quick reference but it will go stale as new tools ship. If an MCP call fails, fall back immediately to the UI/REST steps without interrupting flow.

​

Reference links

• AgentControl quickstart: https://docs.launchdarkly.com/home/ai-configs/quickstart
• AI SDKs overview: https://docs.launchdarkly.com/sdk/ai
• Python AI SDK reference (current, open beta): https://docs.launchdarkly.com/sdk/ai/python
• Node.js AI SDK reference (current, open beta): https://docs.launchdarkly.com/sdk/ai/node-js
• Python AI SDK reference (legacy, maintenance mode): https://docs.launchdarkly.com/sdk/ai/python-legacy
• Node.js AI SDK reference (legacy, maintenance mode): https://docs.launchdarkly.com/sdk/ai/node-js-legacy
• Migrate from the legacy AI SDKs: https://docs.launchdarkly.com/sdk/ai/migration
• Integration guides (written against the legacy AI SDKs): https://docs.launchdarkly.com/guides/ai-configs
• Python observability reference: https://docs.launchdarkly.com/sdk/observability/python
• Node.js observability reference: https://docs.launchdarkly.com/sdk/observability/node-js
• Current AI SDK examples (Python): https://github.com/launchdarkly/python-ai-sdk/tree/main/examples
• Current AI SDK examples (Node.js): https://github.com/launchdarkly/js-ai-sdk/tree/main/examples
• Python sample app (legacy AI SDK): https://github.com/launchdarkly/hello-python-ai
• Node.js sample apps (legacy AI SDK): https://github.com/launchdarkly/js-core/tree/main/packages/sdk/server-ai/examples
• Alpha SDKs (.NET, Go, Ruby): https://docs.launchdarkly.com/sdk/ai
• LaunchDarkly REST API (AgentControl): https://apidocs.launchdarkly.com/tag/AI-configs

​

LaunchDarkly MCP servers

​

Discover available MCP tools at session start

1. List tools on the AgentControl MCP server.
2. List tools on the Feature Management MCP server.
3. If both lists return successfully, prefer MCP for any task they cover. If either probe fails (not installed, auth error, network), fall back to the UI/REST API for that scope without interrupting the user.

​

MCP capability map

• LLM Playground as an interactive browser experience (the *-playground MCP tools cover the data model but the side-by-side interactive comparison UI is browser-only).
• Account-level approval settings (the configuration of when approvals are required — distinct from submitting approval requests).
• Any operation not present in the live tools/list for either server.

​

PHASE 0: DETERMINE STARTING POINT

​

Check for existing app signals

• Source files with AI model calls (.py, .ts, .js)
• Package manifests — package.json, pyproject.toml, requirements.txt, Pipfile
• Imports of AI libraries (OpenAI, Anthropic, LangChain, Bedrock, Gemini, etc.)

​

Decision logic

“I don’t see an existing AI application here. Would you like to:

1. Use a sample app — the fastest way to see LaunchDarkly AgentControl in action, no existing code needed
2. Integrate into an app you’re building — I’ll guide you through setup as you build”

​

SAMPLE APP PATH

Start with the current AI SDK examples, because the sample apps below use the legacy AI SDK. The current examples live at https://github.com/launchdarkly/python-ai-sdk/tree/main/examples for Python and https://github.com/launchdarkly/js-ai-sdk/tree/main/examples for Node.js. Each one has its own README, and they read LD_SDK_KEY rather than LAUNCHDARKLY_SDK_KEY. Use the legacy sample apps below only when an exception in Which AI SDK to use applies.

​

Python sample app

curl -sSL https://install.python-poetry.org | python3 -
# Then restart your shell or run:
export PATH="$HOME/.local/bin:$PATH"

git clone https://github.com/launchdarkly/hello-python-ai
cd hello-python-ai

export LAUNCHDARKLY_SDK_KEY="sdk-..."              # Account settings > Environments in LaunchDarkly UI
export LAUNCHDARKLY_AI_CONFIG_KEY="sample-ai-config"
export OPENAI_API_KEY="sk-..."                     # Or use another provider below

​

Node.js / TypeScript sample apps

git clone https://github.com/launchdarkly/js-core
cd js-core/packages/sdk/server-ai/examples/chat-observability   # recommended: full observability support
npm install

​

PHASE 1: ANALYSIS (read-only)

​

Language gate — check this first

“LaunchDarkly has an alpha AI SDK for [Go/.NET/Ruby] — you can get started with AgentControl, though it currently receives new features at a slower pace than the Python and Node.js SDKs, and does not yet have an observability plugin.

• Go AI SDK: https://docs.launchdarkly.com/sdk/ai/go
• .NET AI SDK: https://docs.launchdarkly.com/sdk/ai/dotnet
• Ruby AI SDK: https://docs.launchdarkly.com/sdk/ai/ruby

Follow the quickstart for your language: https://docs.launchdarkly.com/home/ai-configs/quickstart Would you like to proceed with the alpha SDK, or switch to Python or Node.js for the full experience?”

“LaunchDarkly’s AI SDKs currently support Python, Node.js, Go, .NET, and Ruby. For other languages, you can call the LaunchDarkly REST API directly or use a server-side SDK to evaluate flags. See https://docs.launchdarkly.com/sdk for all SDKs.”

​

Which AI SDK to use, decide this before you install anything

• The app already imports ldai or @launchdarkly/server-sdk-ai. Finish this integration on the SDK that is already wired in, because converting generations is a rewrite rather than a wiring job. Then offer https://docs.launchdarkly.com/sdk/ai/migration as a separate follow-up after the user is connected and seeing data.
• The project is on Python 3.11 or earlier. The current Python AI SDK requires Python 3.12. Do not offer to upgrade the user’s interpreter as part of onboarding.
• The user declines a pre-1.0 dependency. Disclose the beta status up front, as described below. If they say no, use the legacy AI SDK without arguing the point.
• The provider has no handler package, and the user wants neither LangChain nor a custom handler. This covers Strands and the Vercel AI SDK, plus direct boto3 or google-generativeai calls outside LangChain.
• The language is Go, .NET, or Ruby. Those languages have one AI SDK, covered by the language gate above.

“I’ll wire this up with the current LaunchDarkly AI SDK, which is what we recommend for new applications. You register a handler and call config(...).invoke(...), and the SDK calls your model provider and records metrics and traces for you. That is roughly a third of the code of the older SDK. It is in open beta at 0.2.x. I’ll pin exact versions, and APIs can still change between releases. Tell me if you would rather use the stable legacy SDK instead, which is in maintenance mode and receives no new features.”

​

How to scan

1. Check dependency manifests first. These are the most reliable signals:
   • Python: requirements.txt, pyproject.toml, setup.py, Pipfile
   • TypeScript/JavaScript: package.json
2. Scan import statements in source files to confirm what’s in use:
   # Python
   grep -rE "^(import|from)\s+(langchain|langgraph|strands|agents|openai|anthropic|boto3|google)" . \
     --include="*.py" -h | sort -u
   
   # Node.js / TypeScript
   grep -rE "(import|require).*['\"](@langchain|langchain|openai|@anthropic-ai|@aws-sdk|@vercel/ai)" . \
     --include="*.ts" --include="*.js" -h | sort -u
   
3. Check for existing LaunchDarkly setup:
   • ldclient, @launchdarkly/node-server-sdk imports
   • LAUNCHDARKLY_SDK_KEY in .env or config files
   • Existing LDAIClient / LdAiClient usage
4. For monorepos or multi-service projects — ask which service to instrument rather than guessing.
5. Identify the config mode — ask the user if they’re building:
   • Completion mode — a single LLM call per request. The config provides a list of messages (system prompt + optional user/assistant turns) that are sent directly to the model. Good for: chat UIs, summarization, classification, Q&A.
   • Agent mode — multi-step workflows where the model may call tools, loop, or hand off to other agents. The config provides a free-form instructions string (the agent’s goal or persona) rather than a fixed message list. Good for: ReAct loops, LangGraph graphs, OpenAI Agents SDK, Strands.
   If unsure, read a few source files to infer from usage patterns. If the code calls .invoke() / .chat() directly, it is likely completion mode. If it uses a Runner, a tool-calling loop, or a Graph, it is likely agent mode.

​

Phase 1 output

• Detected language, AI framework, and model provider
• Config mode (completion or agent)
• Which AI SDK generation you propose. This is the current AI SDK unless an exception applies, in which case name the exception.
• Proposed LaunchDarkly AI SDK integration (from routing table below), including the handler package when you propose the current AI SDK
• Whether the LaunchDarkly server-side SDK or either AI SDK is already installed

​

INTEGRATION ROUTING TABLE

​

Python

​

TypeScript / JavaScript

​

Handler packages, current AI SDK only

​

Fallback

​

PHASE 2: IMPLEMENTATION

​

1. Fetch the matched integration guide

On the current AI SDK, the integration guides do not apply as written, because they target the legacy AI SDK. Read https://docs.launchdarkly.com/sdk/ai/python or https://docs.launchdarkly.com/sdk/ai/node-js instead, and treat the integration guide as provider reference only.

​

2. Install packages

Scope of this install. Read this before running anything. The only changes that are in-scope without further consent are adding the LaunchDarkly packages named below. Do not upgrade, downgrade, pin, or replace any other packages, even if peer-dependency warnings suggest it. Do not run npm audit fix, pnpm update, poetry update, or any bulk-update command. Do not bump the user’s framework version (LangChain, OpenAI, etc.) “to match” a newer LaunchDarkly SDK. The user may be on an older version on purpose (downstream compatibility, internal pinning, governance policies you cannot see), and silently changing it is a high-cost mistake. If install fails or reports peer conflicts: stop, surface the exact error, and ask the user how to proceed. Use a structured choice:

“The install reported [exact error]. To resolve it I would need to [specific change to non-LD packages]. How would you like to proceed?

1. Yes, make those changes
2. No, keep only the LaunchDarkly packages — I’ll resolve the conflict myself
3. Show me the exact commands first”

Do not write the question as plain text. Present it as a clear choice and wait for an answer. If the user declines, leave their existing dependencies untouched, install only the LaunchDarkly packages if possible, and proceed.

​

Current AI SDK

# Requires Python 3.12 or later; use a virtual environment to avoid system-package conflicts
python3 -m venv .venv && source .venv/bin/activate
pip install launchdarkly-server-sdk "launchdarkly-ai-server[otel]==0.2.2" "launchdarkly-ai-openai-messages==0.2.2" python-dotenv

poetry add launchdarkly-server-sdk "launchdarkly-ai-server[otel]==0.2.2" launchdarkly-ai-openai-messages==0.2.2 python-dotenv

npm install @launchdarkly/ai-node@0.2.0 @launchdarkly/ai-otel@0.1.1 @launchdarkly/ai-openai-messages@0.2.0 dotenv

• Observability is wired differently here. The [otel] extra in Python and the @launchdarkly/ai-otel package in Node.js supply the OpenTelemetry packages the SDK discovers at runtime to export traces. Without them the SDK logs a one-time warning, spans become no-ops, and AI calls otherwise work normally. Do not install launchdarkly-observability or @launchdarkly/observability-node alongside the current AI SDK. The current AI SDK creates its own spans and does not use the observability plugin.
• @launchdarkly/ai-node bundles @launchdarkly/node-server-sdk as a dependency, so do not install the base Node SDK separately. Python works the other way around, so install launchdarkly-server-sdk alongside launchdarkly-ai-server unless the app already initializes a LaunchDarkly client.
• On edge or serverless runtimes such as Vercel or Cloudflare, install the core @launchdarkly/ai-server package instead of @launchdarkly/ai-node, or launchdarkly-ai-server without launchdarkly-server-sdk, and pass your platform’s LaunchDarkly client to initClient() / init_client(client=...) before the first AI call.
• The SDKs read .env through dotenv but do not load it themselves, which is why the dotenv loader is in the install line.

​

Legacy AI SDK

# Use a virtual environment to avoid system-package conflicts
python3 -m venv .venv && source .venv/bin/activate
pip install launchdarkly-server-sdk launchdarkly-server-sdk-ai launchdarkly-observability

poetry add launchdarkly-server-sdk launchdarkly-server-sdk-ai launchdarkly-observability

npm install @launchdarkly/node-server-sdk @launchdarkly/server-sdk-ai @launchdarkly/server-sdk-ai-openai @launchdarkly/observability-node

@launchdarkly/server-sdk-ai-openai provides the getAIMetricsFromResponse helper used in Step 6. Skip it if you are not using OpenAI directly.

​

2.5. Create the config (automated or manual)

Skip this step if you arrived from the LaunchDarkly in-app onboarding — your config was already created and its key came in with the prompt as LAUNCHDARKLY_AI_CONFIG_KEY. The onboarding auto-creates a writing assistant config (key: <handle>s-writing-assistant) with the default model Anthropic.claude-sonnet-4-6. If the user’s provider key is OpenAI (not Anthropic), fix the model immediately — see the troubleshooting row for “wrong model” below.

​

Path A — LaunchDarkly MCP (preferred when connected)

1. Create the config and first variation using setup-ai-config:

   Field	Value
   projectKey	User’s LaunchDarkly project key
   key	Stable identifier, e.g. "my-chatbot"
   name	Human-readable, e.g. "My Chatbot"
   mode	"completion" or "agent" (from Phase 1)
   variationKey	"v1" or "production-initial"
   variationName	"Production (initial)"
   modelConfigKey	"Provider.model-id" — see table in reference section
   modelName	Model identifier string (e.g. "gpt-5.4")
   messages	(completion mode) system/user messages array
   instructions	(agent mode) goal/persona string
   parameters	{"temperature": 0.7, "max_tokens": 2000} etc.

2. Set the default targeting rule using update-rollout (Feature Management MCP):
   • flagKey = the config key (configs are flags under the hood)
   • env = environment key (e.g. "production", "test", "development")
   • rolloutType = "variation", variationIndex = 0
3. Get the SDK key using get-project, unless it came in with the prompt:
   • Use the sdkKey from the matching environment — put it in .env as LAUNCHDARKLY_SDK_KEY

​

Path B — LaunchDarkly UI (always available)

1. Left sidebar → Create → Config → select mode → set name and key → Create
2. Variations tab → fill in model, parameters, and prompt or instructions
3. Targeting tab → Default rule → serve your new variation → Review and save
4. Account settings → Environments → copy the SDK key for your environment, unless it came in with the prompt

​

3. Set up credentials

Tip: If you arrived here from the LaunchDarkly in-app onboarding, LAUNCHDARKLY_SDK_KEY and LAUNCHDARKLY_AI_CONFIG_KEY came in with the prompt. Use those values for the variables below instead of looking them up.

​

Ask before writing any secret — BLOCKING

“Before I add the LaunchDarkly SDK key (and any provider keys), how would you like to set up secrets?

1. Tell me where to put it — give me a file path or secrets-manager command and I’ll write it only there.
2. I’ll set it up myself — just tell me the variable names I need and I’ll handle the values.
3. Write to .env for me — I’ll create or update .env and ensure it’s in .gitignore.”

• Option 1 (Tell me where): ask for the exact path or command. Ask whether the user will paste the key or wants the agent to fetch it via MCP (get-project — see Fetching the SDK key via MCP below). Write the key only to the location they named. Do not create .env or modify any other file.
• Option 2 (I’ll do it myself): list the variable names and the matching LaunchDarkly UI page (Account settings → Environments). Wait for the user to confirm the variables are set before continuing. Do not fetch or write the key value at all.
• Option 3 (Write to .env): ensure .env is listed in .gitignore at the same root before writing any real value (add the entry if missing). Then create or append-update .env with only the LaunchDarkly + provider lines below — never remove unrelated variables. If a .env.example exists, add placeholder entries (no real keys) so teammates know which variables to set.

​

Fetching the SDK key via MCP

​

Variable values

The two SDK generations read different variable names. The current AI SDK reads the SDK key from LD_SDK_KEY, and it labels traces with LD_SERVICE_NAME and LD_ENVIRONMENT. The legacy AI SDK takes the key as an argument, and this prompt passes it from LAUNCHDARKLY_SDK_KEY. If the in-app onboarding handed you LAUNCHDARKLY_SDK_KEY and you are installing the current AI SDK, write the same value to LD_SDK_KEY. Setting both names is fine, and is the safest option in a repo that already uses one of them. A missing LD_SDK_KEY is the most common first-run failure on the current AI SDK. It raises LD_SDK_KEY is not set on the first AI call.

LD_SDK_KEY=sdk-...                    # same value as LAUNCHDARKLY_SDK_KEY
OPENAI_API_KEY=sk-...                 # or ANTHROPIC_API_KEY, GOOGLE_API_KEY, and so on
LD_SERVICE_NAME=my-ai-service         # optional, labels traces
LD_ENVIRONMENT=production             # optional, labels traces

LAUNCHDARKLY_SDK_KEY=sdk-...          # from LaunchDarkly onboarding UI
LAUNCHDARKLY_AI_CONFIG_KEY=your-ai-config-key  # from LaunchDarkly onboarding UI
OPENAI_API_KEY=sk-...
SERVICE_NAME=my-ai-service
SERVICE_VERSION=1.0.0

LAUNCHDARKLY_SDK_KEY=sdk-...
LAUNCHDARKLY_AI_CONFIG_KEY=your-ai-config-key
ANTHROPIC_API_KEY=sk-ant-...
SERVICE_NAME=my-ai-service
SERVICE_VERSION=1.0.0

LAUNCHDARKLY_SDK_KEY=sdk-...
LAUNCHDARKLY_AI_CONFIG_KEY=your-ai-config-key
GOOGLE_API_KEY=...
SERVICE_NAME=my-ai-service
SERVICE_VERSION=1.0.0

​

4. Add the common setup

​

Current AI SDK

import asyncio
from dotenv import load_dotenv
from launchdarkly_ai_server import init_client, shutdown

load_dotenv()

# Optional: pre-warm the client at startup rather than lazily on the first AI call,
# which is worth doing if first-request latency matters, such as serverless cold starts
async def startup() -> None:
    await init_client()

# Call once when the process exits; safe to call more than once
async def teardown() -> None:
    await shutdown()

import 'dotenv/config';
import { initClient, shutdown } from "@launchdarkly/ai-node";

// Optional: pre-warm the client at startup rather than lazily on the first AI call
await initClient();

// Call once when the process exits.
process.on("beforeExit", async () => {
  await shutdown();
});

• The current Python AI SDK is async. Every AI call is awaited, so call sites must be async def, or wrapped in asyncio.run(...).
• The context is a plain object, not a builder. Use {"kind": "user", "key": current_user_id} in Python and { kind: "user", key: currentUserId } in Node.js. Use the real user or session identifier, because that key drives targeting rules, evaluation history, and trace attribution.

​

Legacy AI SDK

import os
import ldclient
from ldclient.config import Config
from ldclient.context import Context
from ldai import LDAIClient, AICompletionConfigDefault, AIAgentConfigDefault, ModelConfig, LDMessage
from ldobserve import ObservabilityConfig, ObservabilityPlugin

ldclient.set_config(Config(
    os.environ["LAUNCHDARKLY_SDK_KEY"],
    plugins=[
        ObservabilityPlugin(
            ObservabilityConfig(
                service_name=os.getenv("SERVICE_NAME", "my-ai-service"),
                service_version=os.getenv("SERVICE_VERSION", "1.0.0"),
            )
        )
    ],
))
aiclient = LDAIClient(ldclient.get())

# Replace with the real user or session identifier (e.g. user.id, session_id, request.user).
# This key drives targeting rules, evaluation history, and trace attribution.
current_user_id = os.getenv("USER_ID", "anonymous")
context = Context.builder(current_user_id).kind("user").build()

import { init, type LDContext } from "@launchdarkly/node-server-sdk";
import { Observability } from "@launchdarkly/observability-node";
import {
  initAi,
  type LDAIClient,
  type LDAIAgentConfig,
  type LDAICompletionConfig,
} from "@launchdarkly/server-sdk-ai";

const ldClient = init(process.env.LAUNCHDARKLY_SDK_KEY!, {
  plugins: [
    new Observability({
      serviceName: process.env.SERVICE_NAME ?? "my-ai-service",
      serviceVersion: process.env.SERVICE_VERSION ?? "1.0.0",
    }),
  ],
});

await ldClient.waitForInitialization({ timeout: 10 });
const aiClient: LDAIClient = initAi(ldClient);

// Replace with the real user or session identifier (e.g. req.user.id, session.id).
// This key drives targeting rules, evaluation history, and trace attribution.
const currentUserId = process.env.USER_ID ?? "anonymous";
const context: LDContext = { kind: "user", key: currentUserId };

​

5. Evaluate the config

​

Current AI SDK

import os
from launchdarkly_ai_server import config
from launchdarkly_ai_openai_messages import create_openai_messages_handler

async def handle_call(user_input: str, current_user_id: str) -> str:
    try:
        result = await config(
            key=os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
            handler=[create_openai_messages_handler()],
            # tool_handlers={"get-user-preferences": get_user_preferences},  # only if the variation defines tools
        ).invoke(
            user_input,
            {"kind": "user", "key": current_user_id},
            {"example_variable": "value"},                     # optional template variables, omit if unused
        )
    except Exception:
        # invoke() raises when the config is disabled, the key is wrong, or LaunchDarkly is
        # unreachable. Return the same hardcoded response the app used before LaunchDarkly.
        return "I'm sorry, this feature is temporarily unavailable."

    return result.response        # model output
    # result.usage               → normalized token counts
    # result.judge_results       → judge scores, when judges are attached to the variation

import { config } from "@launchdarkly/ai-node";
import { createOpenAIHandler } from "@launchdarkly/ai-openai-messages";

async function handleCall(userInput: string, currentUserId: string): Promise<string> {
  try {
    const result = await config({
      key: process.env.LAUNCHDARKLY_AI_CONFIG_KEY!,
      handler: [createOpenAIHandler()],
      // toolHandlers: { "get-user-preferences": getUserPreferences },  // only if the variation defines tools
    }).invoke(
      userInput,
      { kind: "user", key: currentUserId },
      { example_variable: "value" },                     // optional template variables, omit if unused
    );

    return result.response;
  } catch {
    // invoke() throws when the config is disabled, the key is wrong, or LaunchDarkly is unreachable.
    return "I'm sorry, this feature is temporarily unavailable.";
  }
}

The current AI SDK raises instead of returning enabled=False, and it has no default= argument. invoke() throws when the variation has targeting off, when the config key does not exist, or when it cannot reach LaunchDarkly, including during first-time setup before the SDK connects. Two rules follow from that. First, wrap every call path in try/except (or try/catch) that falls back to the exact response the app produced before LaunchDarkly was involved, so an outage cannot take the feature down. Second, if you need to branch on config state without raising, call inspect_config(key, context) in Python or inspectConfig(key, context) in Node.js, which returns {enabled, config, meta} and never raises.

​

Legacy AI SDK

Always provide a default= value. Without one, the SDK returns enabled=False whenever LaunchDarkly is unreachable, including during first-time setup before the SDK connects. The default must duplicate the exact hardcoded values from the original code so behavior is identical during outages. ModelConfig, LDMessage, AICompletionConfigDefault, and AIAgentConfigDefault are imported in Step 4.

config = aiclient.agent_config(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
    default=AIAgentConfigDefault(
        enabled=True,
        model=ModelConfig(name="gpt-5.4"),          # ← your hardcoded model
        instructions="You are a helpful assistant.", # ← your hardcoded prompt
    ),
)
if not config.enabled:
    # config is explicitly disabled in the LaunchDarkly UI.
    return "I'm sorry, this feature is temporarily unavailable."
tracker = config.create_tracker()   # call once per request, after enabled check
# config.instructions    → system prompt / agent goal (str)
# config.model.name      → model identifier (str)
# tracker                → LDAIConfigTracker for metrics (see step 7)

const agentConfig = await aiClient.agentConfig(
  process.env.LAUNCHDARKLY_AI_CONFIG_KEY!,
  context,
  {                                    // default — mirrors your hardcoded values
    enabled: true,
    model: { name: "gpt-5.4" },
    instructions: "You are a helpful assistant.",
  },
);
if (!agentConfig.enabled) {
  // config is explicitly disabled in the LaunchDarkly UI.
  return "I'm sorry, this feature is temporarily unavailable.";
}
const tracker = agentConfig.createTracker();

config = aiclient.completion_config(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
    default=AICompletionConfigDefault(
        enabled=True,
        model=ModelConfig(name="gpt-5.4"),          # ← your hardcoded model
        messages=[
            LDMessage(role="system", content="You are a helpful assistant."),  # ← your hardcoded prompt
        ],
    ),
    variables={"example_variable": "value"},        # optional — omit if not using template variables
)
if not config.enabled:
    # config is explicitly disabled in the LaunchDarkly UI.
    return "I'm sorry, this feature is temporarily unavailable."
tracker = config.create_tracker()   # call once per request, after enabled check
# config.messages            → list[LDMessage] to pass to the model
# config.model.name          → model identifier
# tracker                    → LDAIConfigTracker for metrics

const aiConfig = await aiClient.completionConfig(
  process.env.LAUNCHDARKLY_AI_CONFIG_KEY!,
  context,
  {                                    // default — mirrors your hardcoded values
    enabled: true,
    model: { name: "gpt-5.4" },
    messages: [{ role: "system", content: "You are a helpful assistant." }],
  },
  { example_variable: "value" },       // optional template variables — omit if unused
);
if (!aiConfig.enabled) {
  // config is explicitly disabled in the LaunchDarkly UI.
  return "I'm sorry, this feature is temporarily unavailable.";
}
const tracker = aiConfig.createTracker();

​

6. Add the framework-specific handler

Current AI SDK: skip this step. The handler package is the framework integration. invoke() merges the config’s messages or instructions with the user input, calls the provider through its own SDK, and records metrics and spans. You only write code here if the provider has no handler package, in which case build one with create_handler(provides_for, fn) in Python or createHandler(providesFor, fn) in Node.js, as shown in the SDK reference. Everything below this line is legacy AI SDK code.

Model name pattern: config.model can be None if the config variation has no model configured. Always provide a hard-coded fallback: model_name = config.model.name if config.model else "gpt-5.4". Choose the fallback that matches your stack (e.g. "claude-sonnet-4-6" for Anthropic, "o4-mini" for a cost-optimized OpenAI option).

from openai import OpenAI
from ldai_openai import get_ai_metrics_from_response

openai_client = OpenAI()

def handle_call(config, user_input: str):
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "gpt-5.4"
    # OpenAI spans are emitted automatically by the observability plugin — no decorator needed.
    return tracker.track_metrics_of(
        get_ai_metrics_from_response,
        lambda: openai_client.chat.completions.create(
            model=model_name,
            messages=[m.to_dict() for m in (config.messages or [])] + [{"role": "user", "content": user_input}],
        ),
    )

import { OpenAI } from "openai";
import { getAIMetricsFromResponse } from "@launchdarkly/server-sdk-ai-openai";

const openaiClient = new OpenAI();

async function handleCall(aiConfig: LDAICompletionConfig, userInput: string) {
  const tracker = aiConfig.createTracker();
  return tracker.trackMetricsOf(
    getAIMetricsFromResponse,
    async () => openaiClient.chat.completions.create({
      model: aiConfig.model?.name ?? "gpt-5.4",
      messages: [...(aiConfig.messages ?? []), { role: "user", content: userInput }],
    }),
  );
}

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_community.callbacks import get_openai_callback
from ldai.tracker import TokenUsage

def handle_call(config, user_input: str) -> str:
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "gpt-5.4"
    llm = ChatOpenAI(model=model_name)
    messages = []
    if config.instructions:
        messages.append(SystemMessage(content=config.instructions))
    messages.append(HumanMessage(content=user_input))
    with get_openai_callback() as cb:
        response = llm.invoke(messages)
    tracker.track_tokens(TokenUsage(
        input=cb.prompt_tokens,
        output=cb.completion_tokens,
        total=cb.total_tokens,
    ))
    tracker.track_success()
    return response.content

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, AIMessage, SystemMessage
from langchain_community.callbacks import get_openai_callback
from ldai.tracker import TokenUsage

def handle_call(config, user_input: str) -> str:
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "gpt-5.4"
    llm = ChatOpenAI(model=model_name)
    # config.messages is a list[LDMessage] from the config variation
    lc_messages = []
    for m in (config.messages or []):
        if m.role == "system":
            lc_messages.append(SystemMessage(content=m.content))
        elif m.role == "assistant":
            lc_messages.append(AIMessage(content=m.content))
        else:
            lc_messages.append(HumanMessage(content=m.content))
    lc_messages.append(HumanMessage(content=user_input))
    with get_openai_callback() as cb:
        response = llm.invoke(lc_messages)
    tracker.track_tokens(TokenUsage(
        input=cb.prompt_tokens,
        output=cb.completion_tokens,
        total=cb.total_tokens,
    ))
    tracker.track_success()
    return response.content

from agents import Agent
from agents.run import Runner

async def handle_call(config, user_input: str) -> str:
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "gpt-5.4"
    agent = Agent(name="assistant", instructions=config.instructions or "", model=model_name)
    result = await Runner.run(agent, user_input)
    tracker.track_success()
    return result.final_output

from strands import Agent
from strands.models.openai import OpenAIModel

async def handle_call(config, user_input: str) -> str:
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "gpt-5.4"
    openai_model = OpenAIModel(model_id=model_name)
    agent = Agent(system_prompt=config.instructions or "", model=openai_model, callback_handler=None)
    result = str(agent(user_input))
    tracker.track_success()
    return result

from claude_agent_sdk import query, ClaudeAgentOptions
from claude_agent_sdk.types import ResultMessage

async def handle_call(config, user_input: str) -> str:
    tracker = config.create_tracker()
    model_name = config.model.name if config.model else "claude-sonnet-4-6"
    final_message = None
    async for message in query(
        prompt=user_input,
        options=ClaudeAgentOptions(system_prompt=config.instructions or "", model=model_name),
    ):
        final_message = message
    if not isinstance(final_message, ResultMessage):
        raise ValueError(f"Unexpected message type: {type(final_message)}")
    tracker.track_success()
    return final_message.result or ""

​

7. Track metrics and token usage

Current AI SDK: skip this step. There is no tracker. invoke() and stream() record duration, token counts, success and error, and the OpenTelemetry span for every call. Do not write create_tracker(), track_success(), track_tokens(), or track_metrics_of() against the current AI SDK; those functions do not exist there. The rest of this step is legacy AI SDK code.

from ldai_openai import get_ai_metrics_from_response

tracker = config.create_tracker()
response = tracker.track_metrics_of(
    get_ai_metrics_from_response,
    lambda: openai_client.chat.completions.create(model=..., messages=...),
)

Note: tracker.track_metrics_of(extractor, fn) runs the call, applies the extractor to its response, and records duration, tokens, and success/error in one shot. Every provider goes through track_metrics_of with the appropriate extractor — get_ai_metrics_from_response from ldai_openai for OpenAI, or a small custom extractor for Anthropic, Bedrock, Gemini, and others. See NEXT STEP 11 for extractor examples covering Anthropic, Bedrock, and Gemini.

from ldai.tracker import TokenUsage

tracker = config.create_tracker()
try:
    result = handle_call(config, user_input)        # handler must call tracker.track_success() internally
    # Optionally add token tracking if the framework exposes usage:
    # tracker.track_tokens(TokenUsage(
    #     input=usage.prompt_tokens,
    #     output=usage.completion_tokens,
    #     total=usage.total_tokens,
    # ))
except Exception:
    tracker.track_error()
    raise

Note: track_tokens takes a TokenUsage dataclass (from ldai.tracker import TokenUsage), not a plain dict.

import { getAIMetricsFromResponse } from "@launchdarkly/server-sdk-ai-openai";

const tracker = aiConfig.createTracker();
const response = await tracker.trackMetricsOf(
  getAIMetricsFromResponse,
  async () => openaiClient.chat.completions.create({ model: ..., messages: ... }),
);

const tracker = aiConfig.createTracker();
try {
  const result = await runAgent(agentConfig, userInput);
  tracker.trackTokens({ input: 0, output: 0, total: 0 });  // fill in from your framework
  tracker.trackSuccess();
} catch (e) {
  tracker.trackError();
  throw e;
}

LangChain always exposes token counts via get_openai_callback() — always wrap LangChain calls in that context manager and call tracker.track_tokens() (see the LangChain snippets above). tracker.track_success() alone does not send token data; cost and token metrics in the Monitoring dashboard derive entirely from track_tokens(). For frameworks that genuinely do not expose token counts, omit track_tokens / trackTokens — success/error tracking alone is sufficient to populate request count and error rate.

​

8. Implementation rules

• Read credentials from environment variables. Never hardcode SDK keys or API keys.
• Never cache a resolved config across requests. Evaluate or invoke once per request so targeting and variation changes take effect immediately
• Keep a hardcoded fallback response on every AI call path, so the feature degrades rather than fails when LaunchDarkly is unreachable
• Do not mix the two AI SDK generations in one call path

• Set LD_SDK_KEY; the SDK reads the key from the environment and nowhere else unless you pass sdkKey in options
• Pin the core package and every handler package to exact 0.x versions, all on the same minor version
• Wrap every invoke() in try/except or try/catch; it raises on disabled configs, unknown keys, and connection failures, and there is no default=
• Register one handler per provider and mode the config can serve, or register a wildcard LangChain handler as the fallback
• Do not add a tracker, an observability plugin, or manual spans; metrics and traces are automatic
• Call shutdown() on process exit so events and spans flush
• Python only. Every call is awaited, and call sites must be async.

• Initialize the LaunchDarkly client once at startup, before any agent or model calls
• Always include the observability plugin in the Config/init call, because traces do not appear without it
• Call agent_config() / completion_config() (Python) or agentConfig() / completionConfig() (Node.js) once per request, and never cache the returned config across requests
• Python: Call tracker = config.create_tracker() once per request, after the enabled check, to get the tracker.
• Node.js: Call const tracker = config.createTracker() once per request to get a fresh tracker.
• The observability plugin emits traces automatically, and standard frameworks such as OpenAI and LangChain need no @observe decorator or manual span code
• Always provide a default= argument to completion_config() / agent_config(). Without one, the SDK returns enabled=False when LaunchDarkly is unreachable, including during first-time setup
• Always provide a fallback model name in case config.model is None
• Always call tracker.track_success() or tracker.track_error() after every AI call (or use tracker.track_metrics_of(extractor, fn) / tracker.trackMetricsOf(extractor, fn) which handle this automatically)

​

VERIFICATION

1. Run the application and trigger at least one AI call through the integrated path
2. Check the LaunchDarkly UI. The in-app onboarding shows Connected after the SDK evaluates the config
3. Check the Observability tab. Traces should appear within 1 to 2 minutes of the first call, from the observability plugin on the legacy AI SDK or from the SDK’s own spans on the current AI SDK
4. Check the AgentControl Monitoring tab. Token usage, latency, and success and error rates appear within 1 to 2 minutes of the first tracked call

Set the user’s expectations on data delay. Tell the user up front: “After your first AI call, the Connected state usually flips within seconds, but monitoring data, traces, and judge scores typically take 1–2 minutes to appear in their respective tabs, and sometimes a bit longer. If a tab looks empty right after a call, refresh after a minute or two before troubleshooting.” Saying this once at verification time prevents the very common “I made a call but the page is empty, what’s wrong?” cycle.

​

WHAT’S NEXT

“Your SDK is connected — nice work. Before we go further, head over to your config → Monitoring tab. After a minute or two of AI calls flowing through, you’ll start seeing token usage, latency, and request counts broken down by variation. Make a few AI calls if you haven’t already, give it a moment, and refresh the page. This is where you’ll track the real cost and performance impact of every prompt and model change you make.”

“You just experienced the core value of AgentControl: you changed a prompt or model in the LaunchDarkly UI and your running app picked it up immediately — no redeploy needed. That’s the foundation. Here’s what to explore next:”

11. Complete the migration — extract every remaining hardcoded prompt, model, parameter, and tool into configs in five structured stages

1. Invite your team — give teammates access to edit prompts and models in the LaunchDarkly UI, no code needed
2. Add a judge — automatically score every AI response for accuracy, relevance, and toxicity
3. Run your first eval. Test prompt variations against each other before going to production
4. Review your monitoring data. Token costs, latency, and error rates on the Monitoring tab
5. Log traces. Full request traces linked to config evaluations in the Observability tab
6. Explore more SDK features. Streaming, structured output, registries, and, for legacy AI SDK users, moving to the current AI SDK

7. Agent graphs — orchestrate multi-agent workflows, defined via the AgentControl MCP or the LaunchDarkly UI
8. Run an experiment — A/B test prompt or model variations against real user behavior metrics
9. Guarded rollouts — automatically pause or roll back a model change if quality scores drop
10. Governance and approvals — require review before any config change reaches production

​

NEXT STEP 1: Invite your team

invite-members:
  emails: ["alice@example.com", "bob@example.com"]
  role: "writer"      # or "reader" / "admin"

1. Go to Account settings → Members.
2. Click Invite members.
3. Enter one or more email addresses.
4. Assign a role:
   • Writer — can create and edit configs, variations, targeting rules, and tools. Recommended for anyone who will manage prompts or models.
   • Reader — view-only access. Good for stakeholders who want to review monitoring data without making changes.
   • Admin — full account access, including environment and project settings.
5. Click Send invite. Recipients get an email link to join the LaunchDarkly account.

• Open the config → Variations tab → edit the system prompt or swap the model → Review and save. The change goes live immediately — no deployment needed.
• Use the LLM Playground (top right of the Variations tab) to compare prompt or model options side-by-side before committing.
• Check the Monitoring tab for real-time token costs, latency, and error rates broken down by variation.

​

NEXT STEP 2: Add a judge

On the current AI SDK, attaching a judge to the variation is the whole job. Judges attached to a variation in LaunchDarkly run inline on every invoke() and come back on result.judge_results. No judge code is needed at the call site. If the added latency matters, pass skip_judges=True / skipJudges: true, then hand each task from result.judge_tasks to a background worker that calls run_judge(task, handlers) / runJudge(task, handlers). The code below is for the legacy AI SDK.

​

If completion mode, attach a judge to a variation

update-ai-config-variation:
  projectKey: "my-project"
  configKey: "chat-assistant"
  variationKey: "production-initial"
  judgeConfiguration:
    judges:
      - key: "accuracy"
        sampling: 0.20
      - key: "relevance"
        sampling: 0.20
      - key: "toxicity"
        sampling: 0.20

1. Open your config → Variations tab → click into a variation.
2. In the Judges section, click + Attach judges.
3. Select Accuracy, Relevance, and/or Toxicity. Start at 10–20% sampling to control cost.
4. Click Review and save.

import asyncio
model = await aiclient.create_model(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
)
if not model:
    print("config disabled or unreachable — using fallback")
    # return fallback here
else:
    response = await model.run(user_input)
    print("Response:", response.content)

    # Await judge evaluations before the request ends
    if response.evaluations:
        results = await asyncio.gather(*response.evaluations)
        for r in results:
            print("Judge result:", r.to_dict())

​

If agent mode, invoke a judge directly in code

1. Create a judge config in LaunchDarkly. If MCP is connected, use setup-ai-config with a judge mode and a built-in or custom judge — do this from the agent rather than sending the user to the UI. If MCP is not available, walk the user through AgentControl → Create → choose a built-in judge or custom in the UI.
2. Add its key to your environment: LAUNCHDARKLY_AI_JUDGE_KEY=your-judge-key (use the SDK key consent flow from Phase 2 Step 3 before writing it).

from ldai import AICompletionConfigDefault

judge = await aiclient.create_judge(
    os.environ["LAUNCHDARKLY_AI_JUDGE_KEY"],
    context,
    AICompletionConfigDefault(enabled=False),
)

if judge and judge.enabled:
    result = await judge.evaluate(user_input, agent_response)
    print("Judge score:", result.to_dict())
    # Optionally link the score to your agent's config tracker:
    # tracker.track_judge_result(result)  # tracker = config.create_tracker()

​

Check the Monitoring tab for judge results

“Now head over to your config → Monitoring tab. Scroll down to the User satisfaction section — that’s where judge scores (accuracy, relevance, toxicity) appear as they accumulate. Heads up: judge scores are not instant. Expect a 1–2 minute delay (sometimes a bit more for the very first scores) between making the AI call and seeing the score on this tab. If you don’t see anything yet, that’s almost always the answer — wait a minute or two, refresh the page, and the scores will appear. Once you have data, you can see how scores differ across variations — that’s what makes guarded rollouts and experiments meaningful.”

​

NEXT STEP 3: Run your first eval

# 1. Create a dataset of inputs (and optional expected outputs)
create-dataset:
  projectKey: "my-project"
  key: "qa-baseline"
  rows:
    - input: "What is feature flagging?"
      expected: "..."
    - input: "How does a canary deployment work?"
      expected: "..."

# 2. Create an evaluation that ties the dataset to one or more config variations
create-evaluation:
  projectKey: "my-project"
  key: "v1-vs-v2"
  datasetKey: "qa-baseline"
  configKey: "chat-assistant"
  variationKeys: ["production-initial", "shorter-prompt"]
  judges: ["accuracy", "relevance"]

# 3. Run it and fetch the summary when it's done
run-evaluation:
  projectKey: "my-project"
  evaluationKey: "v1-vs-v2"

get-evaluation-run-summary:
  projectKey: "my-project"
  evaluationKey: "v1-vs-v2"
  runId: "...returned by run-evaluation..."

1. Open your config → click LLM Playground (top right of the Variations tab).
2. Add a second variation (different model or prompt wording).
3. Enter a test input and compare responses side-by-side.
4. For repeatable batch testing: go to Configs → Datasets → New dataset, upload input/output pairs, then run an offline evaluation from the Playground.

judge = await aiclient.create_judge(
    os.environ["LAUNCHDARKLY_AI_JUDGE_KEY"],
    context,
    AICompletionConfigDefault(enabled=False),
)

test_cases = [
    ("What is feature flagging?", expected_answer_1),
    ("How does a canary deployment work?", expected_answer_2),
]

for input_text, expected in test_cases:
    actual = your_model_call(input_text)
    if judge and judge.enabled:
        result = await judge.evaluate(input_text, actual)
        print(f"Score: {result.to_dict()}")

​

NEXT STEP 4: Review your monitoring data

1. Open your config → click the Monitoring tab.
2. Explore the variation-level breakdown if charts already appear, because that means data is flowing.
3. Expect an empty state, or “Waiting for data”, immediately after the first call. Monitoring data, traces, and judge scores typically take 1 to 2 minutes to appear, and sometimes a bit longer for the very first batch. Wait a couple of minutes, then refresh. The data should populate. Tell the user this delay is normal before they start troubleshooting.
4. Check the tracking calls if nothing appears after a few minutes. On the legacy AI SDK, confirm track_success() / track_error() runs after each AI call, as described in Phase 2, Step 7. On the current AI SDK, confirm at least one invoke() returned without raising, because the SDK records metrics per invoke.

from ldai.tracker import FeedbackKind

# tracker was obtained via tracker = config.create_tracker() earlier in the request
tracker.track_feedback({"kind": FeedbackKind.Positive})   # thumbs up
tracker.track_feedback({"kind": FeedbackKind.Negative})   # thumbs down

# At generation time — serialize and return alongside the response
token = tracker.resumption_token
response_payload = {"text": response_text, "ld_token": token}

result = aiclient.create_tracker(token, context)
if result.is_success():
    late_tracker = result.value
    late_tracker.track_feedback({"kind": FeedbackKind.Positive})

tracker.trackFeedback({ kind: LDFeedbackKind.Positive });
// For async feedback: use tracker.resumptionToken and aiClient.createTracker(token, context)

​

NEXT STEP 5: Log traces

1. Run the app and trigger an AI call.
2. Open Observability in the LaunchDarkly left sidebar, then open the Traces tab.
3. Traces appear within 1–2 minutes. If nothing appears after several calls, confirm the ObservabilityPlugin is in the plugins array at init.

from ldobserve import observe  # observe is a module singleton, not a decorator

with observe.start_span("my-agent-call") as span:
    # all AI SDK calls inside this block are linked to this span
    tracker = config.create_tracker()
    result = my_model_call(config, user_input)
    tracker.track_success()

from ldobserve import observe
from opentelemetry import trace

with observe.start_span("custom-llm-call") as span:
    span.set_attribute("gen_ai.request.model", "my-model")
    span.set_attribute("gen_ai.system", "my-provider")
    result = my_custom_llm(prompt)
    span.set_attribute("gen_ai.response.finish_reasons", ["stop"])

​

NEXT STEP 6: Explore more SDK features

• stream() for token-by-token responses, which returns chunk events followed by a done event carrying the full response, usage, and judge results. Streaming and structured output are mutually exclusive.
• Structured output, which you set on the config in the LaunchDarkly UI. When an output format is set, invoke() returns a parsed object as response rather than a string.
• A Registry for handlers and tools, so call sites stop repeating them. Scoped registries are preferred over global_registry because each call site then exposes only the handlers and tools it needs.
• Provider built-in tools, such as Claude’s web search, assigned to a LaunchDarkly tool key so provider-native capabilities still show up in tool-call tracking.
• graph() for multi-agent workflows, covered in NEXT STEP 7.

model = await aiclient.create_model(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
    variables={"username": "Sandy"},
)
if not model:
    # disabled or LD unreachable — return a hard-coded fallback
    return "I'm sorry, this feature is temporarily unavailable."

response = await model.run("Hello, how can you help me?")
print(response.content)
# Token usage, latency, and success tracked automatically — no tracker calls needed

from ldai import AIAgentConfigRequest, AIAgentConfigDefault

agents = aiclient.agent_configs([
    AIAgentConfigRequest(key="summarizer-agent", default=AIAgentConfigDefault(enabled=False)),
    AIAgentConfigRequest(key="validator-agent",  default=AIAgentConfigDefault(enabled=False)),
], context)

summarizer = agents["summarizer-agent"]
validator  = agents["validator-agent"]

​

Reuse common prompt fragments with prompt snippets

create-prompt-snippet:
  projectKey: "my-project"
  key: "company-tone"
  name: "Company tone"
  content: "Respond in a friendly, professional voice. Avoid jargon. Use plural 'we' when describing the company."

list-prompt-snippets / get-prompt-snippet / update-prompt-snippet / delete-prompt-snippet
  # for the rest of the lifecycle

​

NEXT STEP 7: Agent graphs (advanced)

On the current AI SDK, graphs have first-class functions. Call graph(key, handlers=[...]).invoke(input, context, variables) in Python, or graph(key, { handlers: [...] }).invoke(...) in Node.js, to run a graph on the SDK’s model-driven router, which presents each node’s outgoing edges to the model as handoff choices and stops on a final answer, a leaf node, a cycle, or the step cap. Every node emits its own telemetry and runs its own judges. Pass a judge config key as graph_judge / graphJudge to score the graph’s final answer. To hand the topology to a provider’s own runner instead, resolve it with resolve_graph() / resolveGraph() and pass the result to to_openai_agents, to_lang_graph, or to_claude_agents from the matching handler package. The traversal helpers further down this section are for the legacy AI SDK.

create-agent-graph:
  projectKey: "my-project"
  key: "support-triage"
  name: "Support triage"
  rootNodeKey: "router-agent"
  nodes:
    - key: "router-agent"
      configKey: "router-agent-config"
    - key: "billing-agent"
      configKey: "billing-agent-config"
    - key: "tech-agent"
      configKey: "tech-agent-config"
  edges:
    - from: "router-agent"
      to: "billing-agent"
    - from: "router-agent"
      to: "tech-agent"

1. Left sidebar → Configs → Agent graphs → Create agent graph.
2. Add your agent configs as nodes. Assign one as the root.
3. Draw directed edges between nodes to define handoff order and optional handoff data.
4. Save and note the graph key.

graph = aiclient.agent_graph(
    os.environ["LAUNCHDARKLY_GRAPH_KEY"],
    context,
)

def build_agent(node, execution_context):
    cfg = node.get_config()
    model_name = cfg.model.name if cfg.model else "gpt-5.4"
    return your_framework.Agent(
        name=node.get_key(),
        instructions=cfg.instructions or "",
        model=model_name,
    )

# Forward: root → leaf (use when framework builds parents before children)
graph.traverse(build_agent)

# Reverse: leaf → root (use when framework builds children before parents, e.g. LangGraph)
graph.reverse_traverse(build_agent)

​

NEXT STEP 8: Run an experiment (advanced)

# Track a signal that shows the AI response was useful
ldclient.get().track("task-completed", context, metric_value=1)

create-experiment:
  projectKey: "my-project"
  key: "shorter-prompt-test"
  configKey: "chat-assistant"
  variationKeys: ["production-initial", "shorter-prompt"]
  metricKeys: ["task-completed"]
  primaryMetricKey: "task-completed"

start-experiment-iteration:
  projectKey: "my-project"
  experimentKey: "shorter-prompt-test"

1. Go to your config → Targeting tab.
2. Set up a 50/50 percentage rollout between your two variations.
3. Click Review and save → select Start experiment.
4. Choose your metric(s) and set the primary goal.

​

NEXT STEP 9: Guarded rollouts (advanced)

start-guarded-rollout:
  projectKey: "my-project"
  flagKey: "chat-assistant"
  env: "production"
  newVariationKey: "shorter-prompt"
  monitorMetrics: ["accuracy", "relevance"]
  rollbackOnRegression: true

1. Go to your config → Targeting tab.
2. Update the default rule to serve your new variation to an initial percentage of users (e.g., 10%).
3. Click Review and save → in the confirmation modal, select Guarded rollout.
4. Choose the metrics to monitor (judge scores work well here).
5. Set rollback thresholds and enable automatic rollback.
6. Start the rollout.

​

NEXT STEP 10: Governance and approvals (advanced)

1. Go to Account settings → Projects → select your project → select your production environment.
2. Under Approval settings, enable approvals for config changes.
3. Set the minimum number of approvals required and (optionally) restrict who can approve.

​

NEXT STEP 11: Complete the migration (existing-app users)

This is a different migration from the SDK one. These five stages move hardcoded prompts and models out of the user’s code and into AgentControl configs. Moving from the legacy AI SDK to the current AI SDK is a separate piece of work, covered at https://docs.launchdarkly.com/sdk/ai/migration. Do not run both at once. The stage code below is written for the legacy AI SDK; on the current AI SDK the audit and inventory stages are identical, while the wiring stages collapse into config(...).invoke(...) calls with no tracker.

​

Stage 1: Audit — find everything hardcoded

• Model name literals: model="gpt-5.4", model="claude-sonnet-4-6", modelId="anthropic.claude-sonnet-4-6", etc.
• Model parameters: temperature, max_tokens, top_p, max_completion_tokens
• System prompts / instructions: full text of strings passed to system=, systemPrompt:, instructions=, or the first {"role": "system", ...} in a messages array
• Tool definitions: arguments to tools=[...], bind_tools(...), ToolNode(...) — flag each one
• Template placeholders: .format(), f-strings, JS template literals, %(var)s, str.replace("__VAR__", ...) — note each placeholder name, they become {{ variable }} in the config
• Repeated prompt fragments: identical chunks of system prompt or instructions that appear in 2+ call sites — note these for extraction into prompt snippets (one shared fragment, referenced from many variations) in Stage 2.

• Does the app already initialize an LDClient for feature flags? If yes, reuse it — pass it to LDAIClient() / initAi() instead of creating a second one.
• Which config mode (completion or agent) matches how each call site works?

​

Stage 2: Wrap with identical fallback

setup-ai-config:
  projectKey: "my-project"
  key: "chat-assistant"                      ← from audit manifest
  name: "Chat Assistant"
  mode: "completion"                         ← or "agent"
  variationKey: "production-initial"
  variationName: "Production (initial)"
  modelConfigKey: "OpenAI.gpt-5.4"           ← Provider.model-id format
  modelName: "gpt-5.4"
  messages:
    - role: "system"
      content: "You are a helpful assistant."  ← exact hardcoded value
  parameters:
    temperature: 0.7
    max_tokens: 2000

update-rollout:
  projectKey: "my-project"
  flagKey: "chat-assistant"     ← same as the config key
  env: "production"
  rolloutType: "variation"
  variationIndex: 0

1. Left sidebar → Create → AgentControl → select mode → set name and key → Create
2. Variations tab → fill in the exact model, parameters, and system prompt or instructions from your audit manifest. Name the variation “Production (initial)”.
3. Targeting tab → Default rule → serve the new variation → Review and save

from ldai import AICompletionConfigDefault, ModelConfig, LDMessage

FALLBACK = AICompletionConfigDefault(
    enabled=True,
    model=ModelConfig(name="gpt-5.4"),          # exact hardcoded value
    messages=[LDMessage(role="system", content="You are a helpful assistant.")],  # exact hardcoded prompt
)
config = aiclient.completion_config(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
    default=FALLBACK,
)
if not config.enabled:
    return "I'm sorry, this feature is temporarily unavailable."

from ldai import AIAgentConfigDefault, ModelConfig

FALLBACK = AIAgentConfigDefault(
    enabled=True,
    model=ModelConfig(name="gpt-5.4"),
    instructions="You are a helpful assistant.",  # exact hardcoded instructions
)
config = aiclient.agent_config(
    os.environ["LAUNCHDARKLY_AI_CONFIG_KEY"],
    context,
    default=FALLBACK,
)
if not config.enabled:
    return "I'm sorry, this feature is temporarily unavailable."

1. Normal path: response matches pre-migration output
2. Fallback path: unset the SDK key → fallback runs without error, same output
3. Live update: edit the variation in the LaunchDarkly UI, save, rerun → response reflects the change without redeploying

• Fallback duplicates hardcoded values exactly (if it drifts, behavior changes when LaunchDarkly is unreachable)
• Provider call is structurally untouched — only its inputs (model, messages, tools) now come from config
• completion_config / agent_config is called inside the request handler, not at module level at startup

​

Stage 3: Move tools (optional — skip if no function calling)

• LangChain @tool functions: my_tool.args_schema.model_json_schema()
• Plain callables: StructuredTool.from_function(my_fn).args_schema.model_json_schema()
• SDK-native tool definitions: the JSON schema is usually already present in the definition object

create-ai-tool:
  projectKey: "my-project"
  key: "get-weather"
  description: "Get the current weather for a location"
  schema:
    type: "object"
    properties:
      location:
        type: "string"
        description: "City and state, e.g. 'San Francisco, CA'"
    required: ["location"]

update-ai-config-variation:
  projectKey: "my-project"
  configKey: "chat-assistant"
  variationKey: "production-initial"
  tools:
    - key: "get-weather"
      version: 1

​

Stage 4: Instrument the tracker correctly

• Call tracker = config.create_tracker() once per user turn (full request-response cycle, including retries and agent loop iterations) — reuse the same tracker object throughout the turn
• Never share one tracker across unrelated turns; never create a new tracker per loop iteration
• At-most-once methods (track_duration, track_tokens, track_success, track_error) fire once per tracker — a second call logs a warning and no-ops

# At turn start (e.g., entry node)
tracker = config.create_tracker()
total_tokens = TokenUsage(input=0, output=0, total=0)

# Inside the loop — accumulate tool calls and token counts
tracker.track_tool_calls(tool_calls)
# accumulate token usage locally

# At turn end (terminal node, after loop exits)
tracker.track_tokens(total_tokens)
tracker.track_success()   # or tracker.track_error()

from ldai_openai import get_ai_metrics_from_response

tracker = config.create_tracker()
response = tracker.track_metrics_of(
    get_ai_metrics_from_response,
    lambda: openai_client.chat.completions.create(model=..., messages=...),
)
# track_metrics_of handles duration + tokens + success/error automatically

from ldai.providers.types import LDAIMetrics
from ldai.tracker import TokenUsage

def anthropic_extractor(response) -> LDAIMetrics:
    return LDAIMetrics(
        success=response.stop_reason == "end_turn",
        tokens=TokenUsage(
            input=response.usage.input_tokens,
            output=response.usage.output_tokens,
            total=response.usage.input_tokens + response.usage.output_tokens,
        ),
    )

tracker = config.create_tracker()
response = tracker.track_metrics_of(
    anthropic_extractor,
    lambda: anthropic_client.messages.create(...),
)

​

Stage 5: Attach evaluations

​

Guidance for all next steps

• For UI-only topics (account-level approval settings configuration, the interactive LLM Playground browser experience): walk through the UI steps and answer questions. Do not write code unless asked. The UI-only set is shrinking as new MCP tools ship — always check the live tools/list rather than assuming a topic is UI-only. See the MCP capability map for the current reference and the dynamic-discovery directive at the top of the prompt.
• For code topics (judges in code, traces, agent graphs, migration): read the relevant docs URL first, then write the minimal change needed — do not rewrite the entire integration.
• For LaunchDarkly configuration tasks that MCP supports (creating configs, variations, tools, setting targeting, getting SDK keys, submitting approval requests): always prefer MCP when it’s connected — keep the user inside the agent context instead of sending them to the UI. Tell the user what you did via MCP so they can verify in the UI later if they want. Fall back to UI instructions only if MCP is not connected or a call fails. See the MCP capability map.
• Always tailor examples to the user’s language (Python or Node.js) and config mode (completion or agent).
• After any topic is complete, re-offer the next-steps menu. When you do, acknowledge what they just accomplished, reference which steps they’ve already done, and actively recommend the most logical next step rather than simply listing all options again. The goal is to guide the user progressively through the full product — monitoring → judging → experiments → guarded rollouts → governance — so they understand and use each layer, not just the first one they try.
• Keep the momentum going. As users complete more steps, nudge them toward the parts they haven’t explored yet. A user who has added a judge should be encouraged to run their first eval or set up a guarded rollout. A user who has viewed monitoring data should be encouraged to add user satisfaction tracking. Frame each suggestion around what it unlocks for them specifically.
• LaunchDarkly configuration without MCP: The LaunchDarkly UI is always the reliable fallback — it requires no setup and supports every operation covered in this prompt. If the user has an API token, they can also use the REST API (https://app.launchdarkly.com/api/v2, reference: https://apidocs.launchdarkly.com/tag/AI-configs). Never block progress on MCP availability.
• Set delay expectations whenever you point users at a dashboard. Monitoring data, traces, and judge scores typically take 1–2 minutes (sometimes longer for first scores) to populate after the triggering AI call. Tell the user this before they look — it prevents the most common “the dashboard is empty, what’s wrong?” troubleshooting cycle.