Go AI SDK reference

This topic documents how to get started with the Go AI SDK, and links to reference information on all of the supported features.

The Go AI SDK is designed for use with AgentControl. It is in a pre-1.0 release and the API may change based on feedback. You can follow development or contribute on GitHub.

Get started

LaunchDarkly AI SDKs interact with AgentControl configs. Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.

You can use the Go AI SDK to customize your config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDK to record metrics from your AI model generation, including duration and tokens, and to evaluate model output with judges.

Follow these instructions to start using the Go AI SDK in your application.

Install the SDK

First, install the AI SDK as a dependency in your application. How you do this depends on what dependency management system you are using:

• If you are using the standard Go modules system, import the SDK packages in your code and go build will automatically download them. The SDK and its dependencies are modules.
• Otherwise, use the go get command and specify the SDK version, such as go get github.com/launchdarkly/go-server-sdk-ai.

The Go AI SDK is built on the Go SDK, so install that as well.

Here is how:

1
import (
2
    ld "github.com/launchdarkly/go-server-sdk/v7"
3
    "github.com/launchdarkly/go-server-sdk-ai/ldai"
4
)

Initialize the client

After you install and import the SDK, create a single, shared instance of LDClient. Then, use it to initialize the AI client. The AI client is how you interact with configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.

Here is how:

1
client, _ := ld.MakeClient("YOUR_SDK_KEY", 5*time.Second)
2
3
aiClient, err := ldai.NewClient(client)
4
5
if err != nil {
6
    // Client couldn't be created
7
}

This example assumes you have imported the LaunchDarkly SDK package as ld, as shown above.

Configure the context

Next, configure the context that will use the config, that is, the context that will encounter generated AI content in your application. The context attributes determine which variation of the config LaunchDarkly serves to the end user, based on the targeting rules in your config. If you are using template variables in the messages in your config’s variations, the context attributes also fill in values for the template variables.

Here is how:

1
context := ldcontext.NewBuilder("example-context-key").
2
    Kind("user").
3
    Name("Sandy Smith").
4
    SetString("email", "sandy@example.com").
5
    SetValue("groups", ldvalue.ArrayOf(
6
        ldvalue.String("Acme"), ldvalue.String("Global Health Services"))).
7
    Build()

Customize a config

Then, use one of the config retrieval methods to customize a config. Customization means that any variables you include in the messages when you define the config variation have their values set to the context attributes and variables you pass in. The AI SDK provides three config modes, completion, agent, and judge. You set the mode for a particular config when you create it in the LaunchDarkly UI.

The customization process within the AI SDK is similar to evaluating flags in one of LaunchDarkly client-side, server-side, or edge SDKs, in that the SDK completes the customization without a separate network call. If it cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value you provide. For example, you might use an empty, disabled default as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.

All three config modes share a common set of methods through an embedded base: Key(), Enabled(), Model(), ModelName(), Provider(), ProviderName(), and CreateTracker().

Customize configs in completion mode

In completion mode, each variation in your config includes a single set of roles and messages used to prompt your generative AI model. Use CompletionConfig to customize the config.

The CompletionConfig method takes a config key, a context, a fallback value, and optional variables. It performs the evaluation, then returns an AICompletionConfig object with the customized messages and model configuration.

Here is how:

1
fallbackValue := ldai.NewAICompletionConfigDefault().
2
    WithEnabled(true).
3
    WithModelName("claude-sonnet-4-6")
4
5
config := aiClient.CompletionConfig(
6
    "example-config-key",
7
    context,
8
    fallbackValue,
9
    map[string]interface{}{"exampleCustomVariable": "exampleCustomValue"},
10
)

After you call CompletionConfig, you can pass the customized messages directly to your AI provider. To learn more, read Customizing AgentControl configs.

Customize configs in agent mode

In agent mode, use AgentConfig or AgentConfigs to customize the config. The AgentConfig method customizes a single agent config. The AgentConfigs method customizes a batch of them. Agent configs add Instructions() and JudgeConfiguration() on top of the shared base methods.

Here is how:

1
fallbackValue := ldai.NewAIAgentConfigDefault().
2
    WithEnabled(true).
3
    WithModelName("claude-sonnet-4-6")
4
5
agent := aiClient.AgentConfig("example-agent-key", context, fallbackValue, variables)
6
7
instructions := agent.Instructions()

Customize model parameters

Every default value supports builder-style setters that customize the model, provider, and underlying parameters before evaluation:

1
fallbackValue := ldai.NewAICompletionConfigDefault().
2
    WithEnabled(true).
3
    WithModelName("claude-sonnet-4-6").
4
    WithProviderName("anthropic").
5
    WithModelParam("temperature", 0.7).
6
    WithCustomModelParam("top_k", 40).
7
    WithTool(myTool)

Disable a config by default

Provide a fallback value with Enabled set to false so the client falls back to your default behavior if the flag targeting rules do not enable the config, or if LaunchDarkly is unreachable.

Use template configs

CompletionConfigTemplate, AgentConfigTemplate, and JudgeConfigTemplate skip Mustache interpolation and do not accept a variables parameter. Use a template method if you plan to interpolate message content yourself, or if a config has no placeholders to fill.

Evaluate input and output pairs with a judge

Use JudgeConfig to retrieve a judge config. Judge configs add Messages() and EvaluationMetricKey() to the shared base methods.

Here is how:

1
judgeConfig := aiClient.JudgeConfig("example-judge-key", context, fallbackValue, variables)
2
3
type myProvider struct{}
4
5
func (p *myProvider) InvokeStructuredModel(
6
    messages []datamodel.Message,
7
    schema map[string]interface{},
8
) (judge.StructuredResponse, error) {
9
    // Call your AI provider and return a structured response.
10
}
11
12
j, err := judge.New(judgeConfig, tracker, &myProvider{}, "example-judge-key", loggers)

Call Evaluate to score a single input and output pair, or EvaluateMessages to evaluate a full message list against a response. samplingRate is a value between 0.0 and 1.0. If Evaluate skips the call because of sampling, or because the judge config is empty, it returns nil, nil rather than an error:

1
result, err := j.Evaluate(input, output, samplingRate)

Recording the judge response is your responsibility, not the judge’s. Call tracker.TrackJudgeResponse yourself after Evaluate or EvaluateMessages returns. The judge does not call it for you.

Call provider, record metrics from AI model generation

TrackDuration, TrackSuccess, TrackTimeToFirstToken, and TrackTokens are at-most-once, which means LaunchDarkly records only the first call for each. TrackSuccess and TrackError are also mutually exclusive, the first one you call wins. TrackTokens replaces the deprecated TrackUsage method, use TrackTokens in new code.

Here is how:

1
if config.Enabled() {
2
    tracker, err := config.CreateTracker()
3
4
    // Make a request to a provider using details from config.
5
    // For example, you can pass model parameters (config.ModelParam) or messages (config.Messages).
6
7
    tracker.TrackSuccess()
8
    tracker.TrackDuration(elapsed)
9
    tracker.TrackTokens(tokenUsage)
10
} else {
11
    // Application path to take when the config is disabled.
12
}

Each tracker call shares a single runId, a UUIDv4 created when you create the tracker. The runId correlates every event you record on that tracker as one AI run.

Alternatively, you can use TrackDurationOf to wrap a function call and measure its wall-clock duration automatically. For applications that require streaming, use the package-level TrackMetricsOf function to wrap an operation that returns a value and an error. Pass the tracker to TrackMetricsOf along with a function that extracts an AIMetrics value from the operation result. TrackMetricsOf records the operation’s success or error, duration, and tokens, and is the recommended replacement for the deprecated TrackRequest method.

TrackFeedback is multi-fire. Call it as many times as you receive feedback, for example once per user reaction. TrackJudgeResponse is also multi-fire. Use it to record a judge evaluation result.

Use GetSummary to read the metrics recorded on a tracker so far, ResumptionToken to get a token for reconstructing the tracker later, and GetTrackData to read the raw track data.

To learn more, read Tracking AI metrics.

Build and traverse agent graphs

An agent graph links multiple agent configs together into nodes and edges. Retrieve a graph definition with AgentGraph:

1
graph := aiClient.AgentGraph("example-graph-key", context, variables)

AgentGraphDefinition exposes Enabled(), RootNode(), GetNode(key), GetChildNodes(key), GetParentNodes(key), TerminalNodes(), and CreateTracker(), which returns a *GraphTracker.

Each AgentGraphNode exposes Key(), Config(), Edges(), and IsTerminal(). Each GraphEdge exposes the target node’s Key() and a Handoff() map of values passed along that edge.

Traverse and ReverseTraverse walk the graph in topological order. Traverse visits a node only after every reachable predecessor of that node has been visited, starting from the root. ReverseTraverse visits a node only after every reachable descendant has been visited, ending at the root. Both orderings are cycle-safe and deterministic, and give each callback its own dependency-scoped context. LaunchDarkly never mutates the initialContext value you pass in:

1
graph.Traverse(func(node *ldai.AgentGraphNode, context map[string]interface{}) interface{} {
2
    // Run the node.
3
    return result
4
}, initialContext)

GraphTracker records graph-level and edge-level metrics separately. Graph-level methods, such as TrackInvocationSuccess, TrackInvocationFailure, TrackDuration, TrackTotalTokens, and TrackPath, are at-most-once. TrackInvocationSuccess and TrackInvocationFailure are mutually exclusive, so the first call wins, and TrackDuration ignores non-finite values. Edge-level methods, such as TrackHandoffSuccess, TrackHandoffFailure, and TrackRedirect, are multi-fire.

Use GetSummary to read a GraphMetricSummary, and ResumptionToken to get a token for reconstructing the tracker later.

Resume tracking across processes

Both trackers support resumption, so you can start tracking an AI run in one process and continue it in another.

For a completion, agent, or judge tracker, call CreateTracker on the client with a saved token. For a graph tracker, call CreateGraphTracker on the client, or the package-level TrackerGraphFromResumptionToken function directly:

1
tracker, err := aiClient.CreateTracker(token, context)
2
3
graphTracker, err := aiClient.CreateGraphTracker(token, context)

Reconstructing a tracker from a resumption token preserves the original runId, so at-most-once guards still apply across the resumed run.

Supported features

This SDK supports the following features:

• Anonymous contexts
• Context configuration
• Customizing AgentControl configs
• Private attributes
• Tracking AI metrics