.NET AI SDK reference - LaunchDarkly

This SDK is not yet generally availableThe .NET AI SDK is still undergoing testing and active development. Its functionality may change without notice, including becoming backwards incompatible.

This topic documents how to get started with the .NET AI SDK, and links to reference information on all of the supported features. The .NET AI SDK is designed for use with AgentControl. It is in a pre-1.0 release and the API may change based on feedback. Active feature development is ongoing in the Python and Node.js AI SDKs, so this SDK receives new features at a slower pace. You can follow development or contribute on GitHub.

SDK quick linksLaunchDarkly’s SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications:

Resource	Location
SDK API documentation	SDK API docs
GitHub repository	server-ai
Published module	NuGet

For use in server-side applications onlyThis SDK is intended for use in multi-user .NET server applications. To learn more about LaunchDarkly’s different SDK types, read Choosing an SDK type.

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.

Try the QuickstartThis reference guide describes working specifically with the .NET AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with configs, read Quickstart for AgentControl.

You can use the .NET AI SDK to customize your config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDKs to record metrics from your AI model generation, including duration and tokens. Follow these instructions to start using the .NET AI SDK in your application.

Version compatibility

The .NET AI SDK is built for the following targets:

.NET 8.0 and higher
.NET Framework 4.6.2 and higher
.NET Standard 2.0 and higher

The .NET build tools automatically load the most appropriate build for your application’s target platform.

Install the SDK

First, install the AI SDK as a dependency in your application using your application’s dependency manager. If you want to depend on a specific version, refer to the SDK releases page to identify the latest version. The .NET AI SDK is built on the .NET (server-side) SDK, so install both packages. Here’s how:

Install-Package LaunchDarkly.ServerSdk
Install-Package LaunchDarkly.ServerSdk.Ai

Next, import the namespaces in your application code. The namespace is not the same as the package name:

using LaunchDarkly.Sdk.Server.Ai;
using LaunchDarkly.Sdk.Server.Ai.Adapters;
using LaunchDarkly.Sdk.Server.Ai.Config;

Initialize the client

After you install and import the SDK, create a single, shared instance of LdClient. When the LdClient is initialized, use it to initialize the LdAiClient. The LdAiClient is how you interact with configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.

The .NET SDKs use an SDK keyThe .NET AI and server-side SDKs use an SDK key. Keys are specific to each project and environment. They are available on the SDK keys page under Settings. To learn more about key types, read Keys.

Here’s how:

var baseClient = new LdClient(Configuration.Builder("sdk-key-123").StartWaitTime(TimeSpan.FromSeconds(5)).Build());
var aiClient = new LdAiClient(new LdClientAdapter(baseClient));

Configure the context

Next, configure the context that will use the config, that is, the context that will encounter generated AI content in your application. The context attributes determine which variation of the config LaunchDarkly serves to the end user, based on the targeting rules in your config. If you are using template variables in the messages in your config’s variations, the context attributes also fill in values for the template variables. Here’s how:

LDContext context = Context.Builder("example-context-key")
    .Set("firstName", "Sandy")
    .Set("lastName", "Smith")
    .Set("email", "sandy@example.com")
    .Set("groups", LdValue.ArrayOf(LdValue.Of("Acme"), LdValue.Of("Global Health Services")))
    .Build();

Customize the config

Customize your config to set values for variables used in messages, or for instructions based on context attributes or other information you provide. The details of customizing a config depend on whether you are using configs in completion mode or agent mode. You set the mode for a config when you create it in the LaunchDarkly UI.

Customize configs in completion mode

In completion mode, each variation in your config includes a single set of roles and messages used to prompt your AI model. Use CompletionConfig to customize the config. The CompletionConfig method takes an AgentControl config key, a context, a fallback value, and optional variables. It performs the evaluation, then returns an LdAiCompletionConfig object that includes the customized messages and model configuration. Call CreateTracker on the returned object to get a tracker instance you can use to record metrics. If it cannot perform the evaluation or LaunchDarkly is unreachable, the SDK returns the fallback configuration you provided. For example, you might use LdAiCompletionConfigDefault.Disabled as a fallback value, or a fully configured default. Either way, check for this case and handle it appropriately in your application. Here’s how:

var fallbackConfig = LdAiCompletionConfigDefault.Disabled;

var config = aiClient.CompletionConfig(
  "example-config-key",
  context,
  fallbackConfig,
  new Dictionary<string, object> {
    { "exampleCustomVariable", "exampleCustomValue" }
  }
);

var tracker = config.CreateTracker();

Customize configs in agent mode

In agent mode, each variation in your config includes a set of instructions that enable multi-step workflows. In agent mode, use AgentConfig or AgentConfigs to customize the config. The AgentConfig method customizes a single agent config. The AgentConfigs method customizes a list of them. The instructions returned by the SDK come directly from the instructions you define for the variation in the LaunchDarkly UI. The goal or task shown in the UI is delivered unchanged as the Instructions property on the returned object. AgentConfig takes an AgentControl config key, a context, a fallback value, and optional variables. It performs the evaluation, then returns an LdAiAgentConfig object that includes the customized instructions. Call CreateTracker on the returned object to get a tracker instance for recording metrics. AgentConfigs takes IEnumerable<AgentConfigRequest> and returns a dictionary. If the method cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. For example, you might use LdAiAgentConfigDefault.Disabled as a fallback value, or a fully configured default. Either way, check for this case and handle it appropriately in your application. Here’s how:

var fallbackConfig = LdAiAgentConfigDefault.Disabled;

var agent = aiClient.AgentConfig(
  "example-config-key",
  context,
  fallbackConfig,
  new Dictionary<string, object> {
    { "exampleCustomVariable", "exampleCustomValue" }
  }
);

To learn more, read Customizing AgentControl configs.

Evaluate input and output pairs with a judge

The JudgeConfig method takes an AgentControl config key, a context, a fallback value, and optional variables. It returns an LdAiJudgeConfig object that includes the judge’s customized messages, model configuration, and the EvaluationMetricKey to associate scores with. Pass the input and output you want to evaluate as variables so the judge’s message template can reference them. If the SDK cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. You can use LdAiJudgeConfigDefault.Disabled as a fallback, or a fully configured default. Unlike completion mode and agent mode, the .NET AI SDK does not invoke the model for you in judge mode. You are responsible for calling your AI provider with the judge’s messages, parsing the score from the response, and recording it to LaunchDarkly using the tracker’s TrackJudgeResult method. Here’s how:

var fallback = LdAiJudgeConfigDefault.Disabled;

var judge = aiClient.JudgeConfig(
  "example-judge-key",
  context,
  fallback,
  new Dictionary<string, object> {
    { "input", inputText },
    { "output", outputText }
  }
);

if (judge.Enabled) {
  // Call your AI provider using judge.Model and judge.Messages,
  // then parse the score from the response.
  var score = CallModelAndParseScore(judge.Model, judge.Messages);

  // Record the score against the metric key the judge specifies.
  // Use the tracker from the completion or agent config whose output you
  // are evaluating, so the score is associated with that config's run.
  tracker.TrackJudgeResult(new JudgeResult(
    metricKey: judge.EvaluationMetricKey,
    score: score,
    judgeConfigKey: judge.Key
  ));
}

Programmatic judge evaluation does not automatically emit metrics for the config’s Monitoring tab. It also does not attach judges to variations in the LaunchDarkly UI. To learn more, read Online evaluations in AgentControl and Judges.

Call the provider and record metrics from AI model generation

Finally, make a request to your generative AI provider and record metrics from your AI model generation. The steps to do this vary based on whether you use completion mode or agent mode. In completion mode, use the tracker’s TrackMetricsOf method to call your generative AI provider and record metrics from model generation. Check whether the returned LdAiCompletionConfig is enabled and handle the disabled case appropriately in your application. In agent mode, you can access the Instructions returned from the customized config to send to your AI model. Call CreateTracker on the returned LdAiAgentConfig object to get a tracker instance you can use to record metrics. Here’s how:

if (config.Enabled) {

var response = tracker.TrackMetricsOf(
  result => new AiMetrics(success: true, tokens: new Usage(total, input, output)),
  () => CallProviderAsync()
);
    {
      // Make request to a provider, which automatically tracks metrics in LaunchDarkly.
      // When sending the request to a provider, use details from the config.
      // For example, you can pass config.Model and config.Messages.
      // Optionally, return response metadata, for example to do your own additional logging.
      //
      // CAUTION: If the call inside of the function throws an exception,
      // the SDK will re-throw that exception

      return new Response
      {
        Usage = new Usage { Total = 1, Input = 1, Output = 1 }, /* Token usage data */
        Metrics = new Metrics { LatencyMs = 100 } /* Metrics data */
      };
    }
  ));

} else {

  // Application path to take when the config is disabled

}

if (agent.Enabled) {

  var tracker = agent.CreateTracker();

  // Retrieve instructions from the config and pass to your AI model
  var result = ExampleModelApi(agent.Instructions);

  // Track metrics from the result
  tracker.TrackSuccess();

} else {

  // Application path to take when the agent is disabled

}

If you would like to do any tracking beyond what LaunchDarkly provides, you must fill in the AiMetrics object with the data you want to track. Alternatively, you can use the tracker’s other Track* methods to record metrics manually. The tracker provides methods for tracking duration, tokens, time to first token, tool calls, judge results, success, error, and feedback. Make sure to call CompletionConfig or AgentConfig each time you generate content from your AI model. Here’s how:

var config = aiClient.CompletionConfig(
  "example-config-key",
  context,
  fallbackConfig,
  new Dictionary<string, object> {
    { "exampleCustomVariable", "exampleCustomValue" }
  }
);

var tracker = config.CreateTracker();
var response = tracker.TrackMetricsOf(...);

To learn more, read Tracking AI metrics.

Agent graphs

Agent graphs let you define a directed graph of agent configs in the LaunchDarkly UI, then traverse the graph at runtime to orchestrate multi-step workflows. Each node in the graph is an agent config. Each edge defines a handoff from one agent to the next. Use AgentGraph to retrieve a graph by its key. The graph key populates automatically when you create a graph in the LaunchDarkly UI. After the graph exists, the key is in the agent graph details panel. To learn more, read Initialize an agent graph. The method takes a graph key, a context, and optional variables. It returns an AgentGraphDefinition that exposes the graph’s topology and a tracker for recording invocation metrics. Here’s how:

var graph = aiClient.AgentGraph(
  "example-graph-key",
  context,
  new Dictionary<string, object> {
    { "exampleCustomVariable", "exampleCustomValue" }
  }
);

if (graph.Enabled) {
  var tracker = graph.CreateTracker();
  // Walk the graph and invoke each agent's config
}

You can navigate the graph manually using RootNode, GetNode, GetChildNodes, GetParentNodes, and TerminalNodes. Each AgentGraphNode exposes:

Key: the agent config key for this node.
Config: the LdAiAgentConfig for the node, with Instructions, Model, Tools, and other agent config properties.
Edges: the outgoing edges from this node. Each GraphEdge includes the target node Key and a Handoff dictionary describing how to pass state to the next node.
IsTerminal: whether this node has no outgoing edges.

Alternatively, use Traverse or ReverseTraverse to walk the graph with a callback that the SDK invokes for each node. Traverse walks breadth-first from the root. ReverseTraverse walks breadth-first from the terminal nodes back toward the root. Both methods take a callback with the signature Func<AgentGraphNode, Dictionary<string, object>, object> and an optional initial context dictionary. The SDK calls the callback one time for each node, passing the node and a shared context dictionary. The callback’s return value is stored in the context dictionary under the node’s key. This allows downstream nodes to read upstream results. Here’s how:

graph.Traverse((node, context) =>
{
  // Use node.Config to call your AI provider, passing instructions and tools.
  // Read upstream results from context as needed.
  var output = CallModelForNode(node.Config, context);

  // The return value is stored in context[node.Key] for downstream nodes.
  return output;
});

The SDK does not automatically record handoff or invocation metrics during traversal. Call the appropriate AiGraphTracker methods from your callback to record what happens at each node and edge. Use the AiGraphTracker returned by CreateTracker to record metrics for the full graph invocation as well as per-edge handoffs:

TrackInvocationSuccess and TrackInvocationFailure record whether the graph completed successfully
TrackDuration records total invocation duration
TrackTotalTokens records aggregate token usage across all nodes
TrackPath records the sequence of nodes visited
TrackHandoffSuccess and TrackHandoffFailure record per-edge handoff outcomes
TrackRedirect records when a node redirects to a different target than its declared edges

You can call the provider for each node the same way you do in agent mode. To learn more, read Customize configs in agent mode. To learn more about defining agent graphs in the LaunchDarkly UI, read Agent graphs.

Resume tracking across processes

If a single AI workflow spans multiple processes, you can resume tracking on the same config or graph invocation in a different process. An example of this is if a request handler hands work off to a background job. Each tracker exposes a ResumptionToken property. In the original process, read the token after you create the tracker, then pass it to the next process through a job queue, database row, or message envelope. In the receiving process, call aiClient.CreateTracker for a completion or agent tracker, or aiClient.CreateGraphTracker for an agent graph tracker. Both methods take the resumption token and a context and return a tracker that records to the same invocation as the original. Here’s how:

var config = aiClient.CompletionConfig("example-config-key", context, fallbackConfig);
var tracker = config.CreateTracker();

// Hand off work to another process. Persist the resumption token so the
// receiving process can reconstruct the tracker.
var token = tracker.ResumptionToken;
EnqueueBackgroundJob(token, context);

// Reconstruct the tracker using the token and the same context.
var tracker = aiClient.CreateTracker(token, context);

// Continue tracking against the original invocation.
tracker.TrackSuccess();

For agent graphs, use CreateGraphTracker in place of CreateTracker. The token comes from AiGraphTracker.ResumptionToken on the original graph tracker.

Supported features

This SDK supports the following features:

​Get started

​Version compatibility

​Install the SDK

​Initialize the client

​Configure the context

​Customize the config

​Customize configs in completion mode

​Customize configs in agent mode

​Evaluate input and output pairs with a judge

​Call the provider and record metrics from AI model generation

​Agent graphs

​Resume tracking across processes

​Supported features

Get started

Version compatibility

Install the SDK

Initialize the client

Configure the context

Customize the config

Customize configs in completion mode

Customize configs in agent mode

Evaluate input and output pairs with a judge

Call the provider and record metrics from AI model generation

Agent graphs

Resume tracking across processes

Supported features