> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Python AI SDK reference

<View title="Developer" />

<View title="Federal docs" />

<View title="EU docs" />

This topic explains how to get started with the Python AI SDK, and links to reference information on all of the supported features. This SDK is the current AI SDK for Python. It is not a drop-in replacement for the earlier [legacy Python AI SDK](/docs/sdk/ai/python-legacy), which is now in maintenance mode. We recommend that new applications use this SDK. If you are migrating from the legacy SDK, read [Migrate from the legacy AI SDKs](/docs/sdk/ai/migration).

The Python AI SDK provides an opinionated, abstracted entry point to [AgentControl](/docs/home/agentcontrol). Rather than configuring provider routing, client wiring, and message conversion yourself, the SDK handles them for you.

<Warning>
  **The Python AI SDK is in open beta**

  The Python AI SDK is in open beta.

  We recommend pinning the SDK and handler packages you install to specific versions since APIs may have breaking changes during the beta period.

  You can follow development or contribute on [Python AI SDK GitHub repo](https://github.com/launchdarkly/python-ai-sdk).
</Warning>

<Note>
  **SDK quick links**

  LaunchDarkly SDKs are open source. In addition to this reference guide, we provide source, sample applications, and provider-specific packages:

  <table>
    <thead>
      <tr>
        <th>Resource</th>
        <th>Location</th>
      </tr>
    </thead>

    <tbody>
      <tr>
        <td>GitHub repository</td>
        <td>[python-ai-sdk](https://github.com/launchdarkly/python-ai-sdk)</td>
      </tr>

      <tr>
        <td>Sample applications</td>
        <td>[Examples directory](https://github.com/launchdarkly/python-ai-sdk/tree/main/examples)</td>
      </tr>

      <tr>
        <td>Published modules</td>
        <td>[PyPI](https://pypi.org/project/launchdarkly-ai-server/)</td>
      </tr>

      <tr>
        <td>Provider-specific packages</td>
        <td>OpenAI, Anthropic, and LangChain handler packages. Read [Install the SDK](#install-the-sdk).</td>
      </tr>
    </tbody>
  </table>
</Note>

## Get started

LaunchDarkly AI SDKs interact with [AgentControl configs](/docs/home/getting-started/vocabulary#agentcontrol-config). Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications. When you invoke a config with the Python AI SDK, LaunchDarkly controls which model, provider, prompt, and tools are used at runtime. You change providers, models, or prompts in LaunchDarkly without deploying code.

The Python AI SDK follows a three-tier architecture:

1. The **core client** (`launchdarkly-ai-server`) manages the LaunchDarkly lifecycle, telemetry, and the `config()`, `graph()`, and `resolve_graph()` entry-point functions.
2. **Handler packages** integrate a specific provider and mode, such as OpenAI in messages mode or Anthropic in agent mode.
3. Your **application** registers the handlers and tools it needs, then makes calls.

Follow these instructions to start using the Python AI SDK in your application.

### Prerequisites

To use the Python AI SDK, you need:

* Python 3.12 or later.
* A LaunchDarkly account where you have completed the [Quickstart for AgentControl](/docs/home/agentcontrol/quickstart).
* An existing AgentControl config.

### Install the SDK

The SDK installs in up to three parts:

1. Install the base [LaunchDarkly Python SDK](/docs/sdk/server-side/python) by adding `launchdarkly-server-sdk`. Skip this step if your application already initializes a LaunchDarkly client, or if the base SDK does not run on your platform. To learn more, read [Bring your own LaunchDarkly client](#bring-your-own-launchdarkly-client).
2. Install the core AI package by adding `launchdarkly-ai-server`.
3. Install a handler package for each provider and mode you want to use. The following handler packages are available:

<table>
  <thead>
    <tr>
      <th>Package</th>
      <th>Provider</th>
      <th>Mode</th>
      <th>Convenience function</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`launchdarkly-ai-openai-messages`</td>
      <td>OpenAI</td>
      <td>`messages`</td>
      <td>`openai_messages`</td>
    </tr>

    <tr>
      <td>`launchdarkly-ai-openai-agents`</td>
      <td>OpenAI</td>
      <td>`agent`</td>
      <td>`openai_agents`</td>
    </tr>

    <tr>
      <td>`launchdarkly-ai-claude-messages`</td>
      <td>Anthropic</td>
      <td>`messages`</td>
      <td>`claude_messages`</td>
    </tr>

    <tr>
      <td>`launchdarkly-ai-claude-agents`</td>
      <td>Anthropic</td>
      <td>`agent`</td>
      <td>`claude_agents`</td>
    </tr>

    <tr>
      <td>`launchdarkly-ai-langchain-messages`</td>
      <td>Any (`*`)</td>
      <td>`messages`</td>
      <td>`langchain_messages`</td>
    </tr>

    <tr>
      <td>`launchdarkly-ai-langchain-agents`</td>
      <td>Any (`*`)</td>
      <td>`agent`</td>
      <td>`langchain_agents`</td>
    </tr>
  </tbody>
</table>

If your provider does not have a pre-built package, use the LangChain handler package, which works with any provider, or [write a custom handler](#write-a-custom-handler).

Here's how to install the base LaunchDarkly Python SDK, the core AI package, and the OpenAI messages handler:

<CodeGroup>
  ```bash title="Shell" lines wrap theme={null}
  pip install launchdarkly-server-sdk launchdarkly-ai-server launchdarkly-ai-openai-messages
  ```
</CodeGroup>

Replace `launchdarkly-ai-openai-messages` with the handler package for your provider from the table above.

To export traces to LaunchDarkly observability,  add the `otel` extra to `launchdarkly-ai-server`. To learn more, read [Configure observability](#configure-observability).

#### Bring your own LaunchDarkly client

If your application already initializes a LaunchDarkly client, you can hand that client to the AI SDK instead of letting the SDK create one. In this case, omit `launchdarkly-server-sdk` from your install and install only the core AI package and your handler packages.

Here's how to install the core AI package and the OpenAI messages handler without the base Python SDK:

<CodeGroup>
  ```bash title="Shell" lines wrap theme={null}
  pip install launchdarkly-ai-server launchdarkly-ai-openai-messages
  ```
</CodeGroup>

Then pass your client to `init_client()` as the `client` keyword argument before your first AI call:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import init_client

  # my_client is a LaunchDarkly client your application already initialized
  await init_client(client=my_client)
  ```
</CodeGroup>

The SDK uses the client you supply for all config evaluation and event tracking. You are responsible for that client's configuration and lifecycle. To learn more, read [Manage the client lifecycle](#manage-the-client-lifecycle).

#### Edge and custom runtimes

The base LaunchDarkly Python SDK doesn't run on every platform. If your platform restricts package installation or limits execution time, such as an edge or serverless runtime, install `launchdarkly-ai-server` without `launchdarkly-server-sdk` and pair it with the LaunchDarkly SDK for your platform instead. Then pass your platform's client to `init_client()` as described in [Bring your own LaunchDarkly client](#bring-your-own-launchdarkly-client).

### Configure the environment

The SDK reads your LaunchDarkly SDK key and your provider API keys from environment variables. Set `LD_SDK_KEY` and the API key for each provider you use, such as `OPENAI_API_KEY` or `ANTHROPIC_API_KEY`.

The SDK also reads optional environment variables for connection routing, including the following:

* `LD_BASE_URI`
* `LD_STREAM_URI`
* `LD_EVENTS_URI`

It also reads optional variables for observability. To learn more about observability variables, read the [Configure observability](#configure-observability) section.

Your SDK key is specific to each project and environment. Find it by going to your LaunchDarkly **Settings** and locating the **SDK keys** page.

How you set these depends on your setup. Common options include a `.env` file, your shell, or your platform's environment configuration UI. Many secrets managers can inject stored values as environment variables too.

The examples that follow use a `.env` file loaded with `load_dotenv()`.

### Call a model with a convenience function

A convenience function is a single, pre-wired call. It's the most opinionated entry point and intentionally trades configurability for speed. You can swap a provider SDK call for the LaunchDarkly equivalent and get the same behavior as well as targeting, metrics, and prompt management.

Each handler package exports one convenience function, the shortest path to a working call. Import it and call it with the AgentControl config key, the user input, and an [LDContext](/docs/home/getting-started/vocabulary#context), in that order. Behind the scenes, the SDK evaluates the config, selects the provider, calls the model, and records metrics.

To interpolate values into your config's messages, pass an optional `variables` keyword argument.

Here's how to call an OpenAI config:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from dotenv import load_dotenv
  from launchdarkly_ai_openai_messages import openai_messages

  load_dotenv()

  result = await openai_messages(
      "my-ai-config",                         # AgentControl config key: which LD config to use
      "What is feature flagging?",            # User input
      {"kind": "user", "key": "user-123"},    # User context: identifies who is making this request
      variables={"my_variable": "my-value"},  # Optional: variables to interpolate into your messages
  )

  print(result.response)
  ```
</CodeGroup>

### Call a model with a handler

For more control, use `config()` from the core package. Unlike a convenience function, `config()` gives you routing across multiple providers, custom variables, tools, streaming, and async judges. The `config()` function accepts an AgentControl config key and one or more handlers, then routes to the correct handler at invocation time based on the config variation's provider and mode. It returns an object with the `invoke()` and `stream()` methods.

Call `invoke()` with the user input, an [LDContext](/docs/home/getting-started/vocabulary#context), and an optional map of variables. The SDK uses the variables to fill in `{{variable}}` placeholders in the config's messages or instructions. It also injects the LDContext under the `ld_context` key, so templates can reference context attributes with `{{ld_context.key}}` or `{{ld_context.email}}`.

<Warning>
  **invoke() raises on a disabled or unreachable config**

  There is no fallback argument in this SDK. `invoke()` raises an error if the variation has targeting off, if the config key does not exist, or if the SDK cannot reach LaunchDarkly. To prevent this, wrap the call sites and return a fallback response.

  To read the state without raising, call `inspect_config(key, context)`. It returns the `enabled` flag, the parsed config, and the variation metadata. It returns a result even for a config with targeting off or one the SDK cannot reach.
</Warning>

Here's how to route between multiple providers:

<CodeGroup>
  ```python title="Python" expandable lines wrap theme={null}
  from dotenv import load_dotenv
  from launchdarkly_ai_server import config, shutdown
  from launchdarkly_ai_openai_messages import create_openai_messages_handler
  from launchdarkly_ai_openai_agents import create_openai_agent_handler
  from launchdarkly_ai_claude_messages import create_claude_messages_handler
  from launchdarkly_ai_claude_agents import create_claude_agents_handler
  from launchdarkly_ai_langchain_messages import create_langchain_messages_handler

  load_dotenv()

  result = await config(
      key="my-ai-config",         # AgentControl config key
      handler=[                   # Handlers for each provider and mode you support
          create_openai_messages_handler(),
          create_openai_agent_handler(),
          create_claude_messages_handler(),
          create_claude_agents_handler(),
          create_langchain_messages_handler(),
      ],
  ).invoke(
      "What is feature flagging?",           # User input
      {"kind": "user", "key": "user-123"},   # User context
      {"user_name": "Ada"},                  # Optional variables for the prompt template
  )

  print(result.response)

  # Flush pending LaunchDarkly events and OpenTelemetry spans, then close the connection
  await shutdown()
  ```
</CodeGroup>

<Tip>
  **LangChain handlers match any provider**

  The LangChain handler packages register with the wildcard provider `'*'`. The router treats `'*'` as a fallback that matches any provider when no handler with an exact provider name is present. Handlers with an explicit provider name always take precedence over the wildcard.
</Tip>

#### Write a custom handler

If you use a provider that does not have a pre-built package, or you want an internal endpoint, use `create_handler()` to build your own. The `create_handler(provides_for, fn)` function attaches routing metadata to a function so it works with `config()` routing and registries.

You can also use a custom handler as a fallback for providers or models you do not support. Register a handler that returns an unsupported-provider response, and `config()` routes unmatched calls to it.

Here's how to write a custom handler:

<CodeGroup>
  ```python title="Python" expandable lines wrap theme={null}
  import httpx
  from launchdarkly_ai_server import config, create_handler

  async def _call_internal(ld_config, user_input, tool_handlers, variables):
      # The SDK passes in ld_config (the resolved LD config) and user_input (the caller's request).
      async with httpx.AsyncClient() as client:
          response = await client.post(
              "https://models.internal.example.com/generate",
              json={"model": ld_config.model.name, "prompt": user_input},
          )
      data = response.json()
      return {"output": data["text"], "usage": data["usage"]}

  internal_handler = create_handler(["InternalProvider", "messages"], _call_internal)

  result = await config(
      key="my-ai-config",
      handler=internal_handler,
  ).invoke("What is feature flagging?", {"kind": "user", "key": "user-123"})
  ```
</CodeGroup>

### Manage handlers and tools with a registry

Rather than passing handlers and tools to every call, you can register them once and reuse them. A `Registry` is a container with two properties:

* `handlers`: A list of handler objects created by provider handler functions.
* `tools`: A dictionary whose keys are tool names and whose values are the functions that run when the model calls each tool.

A registry can hold tools for many different configs. When you use a registry with `config()`, the SDK only passes the tools that the resolved config variation names, not every tool in the registry. The registry defines which tools are available for selection, not which tools an individual call receives.

You pass the registry as the `registry` option to any call.

<Warning>
  **Tool keys must match exactly**

  Tool keys in the registry must match the tool names defined in the config variation exactly, including case. A mismatch causes a runtime error when the model requests the tool.
</Warning>

#### Use a scoped registry

We recommend using scoped registries over the global registry, because they follow the [principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege). Each registry exposes only the handlers and tools a given call site needs. Use a scoped registry when different parts of your application need different handlers or tools.

Here's how to create and use a scoped registry:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import Registry, config
  from launchdarkly_ai_openai_messages import create_openai_messages_handler
  from launchdarkly_ai_openai_agents import create_openai_agent_handler

  openai_registry = Registry(
      handlers=[create_openai_messages_handler(), create_openai_agent_handler()],
      tools={"get-prefs": get_preferences_fn},
  )

  result = await config(key="my-ai-config", registry=openai_registry).invoke(user_input, context)
  ```
</CodeGroup>

#### Use the global registry

The `global_registry` is a process-wide singleton exported from the core package. It already exists, so you populate it with `register()` rather than constructing a `Registry` yourself. Call `register()` one time at startup, then reference the registry from any call site. Use the global registry only when a single set of handlers and tools is appropriate for your whole application.

Here's how to populate and use the global registry:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import global_registry, config
  from launchdarkly_ai_openai_messages import create_openai_messages_handler
  from launchdarkly_ai_claude_agents import create_claude_agents_handler, ClaudeWebSearch

  # Call once at application startup
  global_registry.register(
      handlers=[create_openai_messages_handler(), create_claude_agents_handler()],
      tools={
          "web-search": ClaudeWebSearch,
          "get-prefs": get_preferences_fn,
      },
  )

  # Reference the registry from any call site
  result = await config(key="my-ai-config", registry=global_registry).invoke(user_input, context)
  ```
</CodeGroup>

You can combine two registries with `compose(a, b)`. For example, you might layer call-site-specific handlers and tools on top of a shared base registry rather than redefining them. It returns a new third registry and does not modify either input.

When a handler key or tool name appears in both, the second registry takes precedence, so later registries override earlier ones. `compose()` works on any two registries, whether scoped, global, or a mix.

You can also pass `handler` and `tool_handlers` inline on a single call, even when you supply a registry. Inline options always take precedence over the registry, so you can override a handler or tool for one call without changing shared configuration.

### Use built-in provider tools

Some provider SDKs expose built-in capabilities that the provider handles natively rather than dispatching to a function you write. For example, the Claude Agent SDK has a built-in tool for web search. To use one, assign the built-in tool as the implementation for a LaunchDarkly tool key in place of a function, as with `ClaudeWebSearch` in the following example. Assigning a LaunchDarkly tool key lets you use the provider's built-in tools with LaunchDarkly features such as tool-call tracking.

Here's how to enable the Claude web search tool:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import config
  from launchdarkly_ai_claude_agents import create_claude_agents_handler, ClaudeWebSearch

  result = await config(
      key="my-ai-config",
      tool_handlers={
          "web-search": ClaudeWebSearch,           # Built-in tool, no function needed
          "get-prefs": get_preferences_fn,
      },
      handler=[create_claude_agents_handler()],
  ).invoke("What are the latest LaunchDarkly release notes?", {"kind": "user", "key": "user-123"})
  ```
</CodeGroup>

The `launchdarkly-ai-claude-agents` package exports additional built-in tools beyond web search, such as file and shell operations. For the current set, read the package documentation for your preferred provider.

### Run an agent graph

An [agent graph](/docs/home/agentcontrol/agent-graphs) is a multi-agent workflow defined in a LaunchDarkly graph object as a root agent plus directed edges. Use the `graph()` function to run one.

The SDK uses a model-driven router. It starts at the root node and presents the outgoing edges to the model as handoff choices. When the model selects an edge, the router follows it, passing the original user request and the previous node's response to the next node.

The loop ends when any of these conditions is met:

* The model produces a final answer.
* Execution reaches a leaf node, a node with no outgoing edges where the graph ends.
* The router detects a cycle.
* Execution hits the step cap.

Each node runs through the same path as `config()`, so every node emits its own telemetry and runs its own judges, tagged with the graph key. Because routing is per node, you can build mixed graphs in which different nodes use different providers.

Here's how to run a graph:

<CodeGroup>
  ```python title="Python" expandable lines wrap theme={null}
  from dotenv import load_dotenv
  from launchdarkly_ai_server import graph, shutdown
  from launchdarkly_ai_claude_agents import create_claude_agents_handler
  from launchdarkly_ai_openai_agents import create_openai_agent_handler

  load_dotenv()

  result = await graph(
      "support-graph",
      handlers=[create_claude_agents_handler(), create_openai_agent_handler()],
  ).invoke(
      "I was double charged",
      {"kind": "user", "key": "user-123"},
      {"account_tier": "pro"},
  )

  print(result["response"])  # Final output
  print(result["usage"])     # Aggregate token counts

  await shutdown()
  ```
</CodeGroup>

To evaluate the final graph output with a judge, pass a judge config key as the `graph_judge` option to `graph()`. Use it when you want a single quality or safety check on the graph's final answer, separate from any judges that run on individual nodes.

The handler packages also export single-provider convenience functions: `claude_graph`, `openai_graph`, and `langchain_graph`. Each pre-binds its own handler and runs the graph on that provider's own agent or graph SDK rather than the model-driven router described above. For example, `openai_graph` uses the OpenAI Agents SDK's own harness. To learn more, read [Run a graph on a native framework runner](#run-a-graph-on-a-native-framework-runner). For mixed-provider graphs, or when you want the router's model-driven handoff behavior, use the base `graph()` function and pass multiple handlers.

#### Run a graph on a native framework runner

If you want a provider's own framework to manage handoffs, tool loops, and conversation state, use the `resolve_graph()` function to resolve the topology without executing it, then pass the result to a native runner. Each handler package ships a runner that converts the resolved graph into the native agent and handoff constructs the provider's framework uses.

The following native runners are available:

<table>
  <thead>
    <tr>
      <th>Runner</th>
      <th>Package</th>
      <th>Framework</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`to_openai_agents`</td>
      <td>`launchdarkly-ai-openai-agents`</td>
      <td>OpenAI Agents SDK</td>
    </tr>

    <tr>
      <td>`to_lang_graph`</td>
      <td>`launchdarkly-ai-langchain-agents`</td>
      <td>LangGraph `StateGraph`</td>
    </tr>

    <tr>
      <td>`to_claude_agents`</td>
      <td>`launchdarkly-ai-claude-agents`</td>
      <td>Claude Agent SDK</td>
    </tr>
  </tbody>
</table>

Here's how to run a graph on LangGraph:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import resolve_graph
  from launchdarkly_ai_langchain_agents import to_lang_graph

  context = {"kind": "user", "key": "user-123"}

  result = await to_lang_graph(
      resolve_graph("support-graph", context=context),
      tool_handlers=registry.tools,
      context=context,
  ).ainvoke("I was double charged")
  ```
</CodeGroup>

The `.ainvoke()` method is LangGraph's own async method, not a LaunchDarkly function. The `a` prefix is LangGraph's convention for async calls.

Native runners bypass the SDK's model-driven router, but they still accept the same `tool_handlers` map as `graph()`, emit the same `$ld:ai:*` tracking events, and wrap execution in a parent OpenTelemetry span.

### Use structured output

A config can return structured output that conforms to a JSON schema, rather than a plain string.

Use structured output when:

* The response needs to be machine-readable, not just human-readable.
* You feed model output into another system, database, or function.
* You need to extract specific structured information, such as a severity or category.
* You build pipelines where the output of one call feeds the input of another.

Set the output format on the config in the LaunchDarkly UI. When an output format is set, the SDK applies it to your call automatically, and `invoke()` returns a parsed object as `response` instead of a string.

For providers with native structured output support, such as OpenAI in messages mode, the SDK uses the provider's built-in structured output feature. For all other providers, the SDK appends schema instructions to the prompt on a best-effort basis. If the model returns output that cannot be parsed, `invoke()` returns the raw string rather than raising an exception.

### Stream a response

Streaming delivers the model's response incrementally as it generates, rather than waiting for the complete response.

To receive a response token by token, use `stream()` instead of `invoke()`. The `stream()` method returns an async generator that yields a `chunk` event for each text delta, followed by a single `done` event that carries the full response, normalized token usage, and any judge results.

Here's how to stream a response:

<CodeGroup>
  ```python title="Python" lines wrap theme={null}
  from launchdarkly_ai_server import config, global_registry

  async for event in config(key="my-ai-config", registry=global_registry).stream(
      "What is feature flagging?",
      {"kind": "user", "key": "user-123"},
  ):
      if event["type"] == "chunk":
          print(event["text"], end="", flush=True)
      else:
          # Final event: full response, usage, and judge results.
          print("\n", event["usage"])
  ```
</CodeGroup>

<Warning>
  **Structured output and streaming are mutually exclusive**

  You cannot use structured output and streaming at the same time. Streaming ignores any output format set on the config. To receive structured output, use `invoke()` rather than `stream()`.
</Warning>

### Run judges asynchronously

Judges evaluate a model's output against criteria you define in a judge config. By default, judges configured on a config run inline, which adds their evaluation time to your response latency. To run judges without increasing latency, evaluate them in a background task.

Pass `skip_judges=True` to `config()` to suppress inline evaluation. `invoke()` then returns `judge_tasks`, a list of pre-packaged, serializable tasks, alongside the model response. Schedule each task on a background worker (a task that runs independently without blocking your main application) and call `run_judge(task, handlers)` inside the worker. Because each task carries everything the judge needs, the worker handles the AI call and the LaunchDarkly tracking event on its own, and your main thread does not block.

You can use any async mechanism you choose. This example uses `asyncio.create_task`, but a thread, a task queue, or an external worker such as a Lambda works too.

Here's how to return judge tasks and hand them to background workers:

<CodeGroup>
  ```python title="Main coroutine (Python)" lines wrap theme={null}
  import asyncio
  from launchdarkly_ai_server import config, global_registry

  result = await config(
      key="my-ai-config",
      registry=global_registry,
      skip_judges=True,
  ).invoke(user_input, context)

  # Fire and forget: the main coroutine continues immediately
  for task in result.judge_tasks or []:
      asyncio.create_task(run_judge_worker(task))
  ```

  ```python title="Worker coroutine (Python)" expandable lines wrap theme={null}
  from launchdarkly_ai_server import get_client, run_judge
  from launchdarkly_ai_openai_messages import create_openai_messages_handler

  async def run_judge_worker(task):
      result = await run_judge(task, [create_openai_messages_handler()])

      if result and task.evaluation_metric_key:
          get_client().track(
              task.evaluation_metric_key,
              task.user_context,
              result.track_data,
              result.score,
          )
  ```
</CodeGroup>

This example runs judges in the same process, so the worker reuses the client the SDK already initialized. If you run judges in a separate process, such as a Lambda or a queue consumer, call `init_client()` in that process before the first judge runs and `shutdown()` before the process exits.

### Configure observability

The Python AI SDK records AI metrics, such as duration and token counts, automatically on every call. It also creates OpenTelemetry spans that follow the [Gen AI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). You do not need to initialize a tracker or wrap your model calls in spans yourself.

To export traces to LaunchDarkly Observability, make the OpenTelemetry SDK packages available at runtime. You have two options:

* Install the `otel` extra, which pulls in the OpenTelemetry packages the client needs. The client discovers them automatically.
* Configure your own OpenTelemetry setup. The client uses the OpenTelemetry packages already present in your environment.

Here's how to install the core AI package with the `otel` extra, alongside the base LaunchDarkly Python SDK and the OpenAI messages handler:

<CodeGroup>
  ```bash title="Shell" lines wrap theme={null}
  pip install launchdarkly-server-sdk "launchdarkly-ai-server[otel]" launchdarkly-ai-openai-messages
  ```
</CodeGroup>

At runtime, the client detects whether the OpenTelemetry packages are present. If they are, it configures a tracer provider with a GZIP-compressed OpenTelemetry Protocol (OTLP) HTTP exporter and W3C trace-context propagators. If they are absent, it logs a one-time warning with the install command and continues. Feature flags and AI calls work normally, and spans become no-ops.

The following environment variables configure telemetry:

<table>
  <thead>
    <tr>
      <th>Variable</th>
      <th>Description</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`LD_SERVICE_NAME`</td>
      <td>The OpenTelemetry `service.name` resource attribute. Defaults to `python-sdk`.</td>
    </tr>

    <tr>
      <td>`LD_ENVIRONMENT`</td>
      <td>The `deployment.environment` resource attribute, such as `production` or `staging`.</td>
    </tr>

    <tr>
      <td>`OTEL_EXPORTER_OTLP_ENDPOINT`</td>
      <td>An override for the OTLP endpoint. Defaults to the LaunchDarkly Observability backend.</td>
    </tr>
  </tbody>
</table>

To learn more, read [LLM observability](/docs/home/observability/llm-observability) and [Monitor configs](/docs/home/agentcontrol/monitor).

### Manage the client lifecycle

The Python AI SDK manages a single shared LaunchDarkly client for you. In most applications, the client initializes lazily on your first AI call, so you do not need to call `init_client()` explicitly. Lazy initialization requires the base LaunchDarkly Python SDK, `launchdarkly-server-sdk`, to be installed. To learn more, read [Install the SDK](#install-the-sdk).

Call `init_client()` yourself when you want to do the following:

* Pass custom initialization options.
* Supply a pre-initialized client, either because your application already manages one or because you run in an edge or custom runtime. Call `init_client(client=my_client)` before your first AI call. To learn more, read [Bring your own LaunchDarkly client](#bring-your-own-launchdarkly-client).
* Pre-warm the client rather than on the first request. Pre-warming initializes it at startup so it's ready before your first user request arrives. We recommend pre-warming the client when first-request latency matters, such as for predictable traffic spikes or serverless cold starts.

Call `shutdown()` when your application exits to flush pending LaunchDarkly events and OpenTelemetry spans and close the connection. `shutdown()` is safe to call more than once.

## Supported features

The Python AI SDK builds on the [Python SDK](/docs/sdk/server-side/python) and supports the following features:

* [Anonymous contexts](/docs/sdk/features/anonymous)
* [Context configuration](/docs/sdk/features/context-config)
* [Customizing AgentControl configs](/docs/sdk/features/agentcontrol-config)
* [Private attributes](/docs/sdk/features/private-attributes)
* [Tracking AI metrics](/docs/sdk/features/ai-metrics)

## What's next

Now that you can call models with the Python AI SDK, explore these next steps:

* [Create configs](/docs/home/agentcontrol/create) to set up your own AgentControl configs in the LaunchDarkly UI.
* [Agent graphs](/docs/home/agentcontrol/agent-graphs) to build multi-agent workflows.
