Node.js (server-side) AI SDK reference
Overview
This topic documents how to get started with the Node.js (server-side) AI SDK, and links to reference information on all of the supported features. You can use either JavaScript or TypeScript when working with the Node.js (server-side) AI SDK.
The Node.js (server-side) AI SDK is designed for use with LaunchDarkly’s AI Configs. It is currently in a pre-1.0 release and under active development. You can follow development or contribute on GitHub.
SDK quick links
LaunchDarkly’s SDKs are open source. In addition to this reference guide, we provide source, API reference documentation, and sample applications:
For use in server-side applications only
This SDK is intended for use in multi-user Node.js server applications. To learn more about LaunchDarkly’s different SDK types, read Client-side, server-side, and edge SDKs.
Get started
LaunchDarkly AI SDKs interact with AI Configs. AI Configs are the LaunchDarkly resources that manage model configurations and messages for your generative AI applications.
Try the Quickstart
This reference guide describes working specifically with the Node.js (server-side) AI SDK. For a complete introduction to LaunchDarkly AI SDKs and how they interact with AI Configs, read Quickstart for AI Configs.
You can use the Node.js (server-side) AI SDK to customize your AI Config based on the context that you provide. This means both the messages and the model evaluation in your generative AI application are specific to each end user, at runtime. You can also use the AI SDKs to record metrics from your AI model generation, including duration and tokens.
Follow these instructions to start using the Node.js (server-side) AI SDK in your application.
Install the SDK
First, install the AI SDK as a dependency in your application using your application’s dependency manager. If you want to depend on a specific version, refer to the SDK releases page to identify the latest version. The Node.js (server-side) AI SDK is built on the Node.js (server-side) SDK, so you’ll need to install that as well.
Here’s how:
Next, import init
, LDContext
, and initAi
in your application code. If you are using TypeScript, you can optionally import the LaunchDarkly LDAIClient
and LDAIConfig
. These are implied, so are not strictly required.
Here’s how:
Initialize the client
After you install and import the SDK, create a single, shared instance of LDClient
. When the LDClient
is initialized, use it to initialize the LDAIClient
. The LDAIClient
is how you interact with AI Configs. Specify the SDK key to authorize your application to connect to a particular environment within LaunchDarkly.
The Node.js SDKs use an SDK key
Both the Node.js (server-side) AI SDK and the Node.js (server-side) SDK use an SDK key. Keys are specific to each project and environment. They are available from Project settings, on the Environments list. To learn more about key types, read Keys.
Here’s how:
Configure the context
Next, configure the context that will use the AI Config, that is, the context that will encounter generated AI content in your application. The context attributes determine which variation of the AI Config LaunchDarkly serves to the end user, based on the targeting rules in your AI Config. If you are using template variables in the messages in your AI Config’s variations, the context attributes also fill in values for the template variables.
Here’s how:
Customize an AI Config
The next step is to customize your AI Config. Customization means that any variables you include in the messages or instructions when you define the AI Config variation have their values set to the context attributes and variables you pass in.
The details of customizing an AI Config depend on whether you are using AI Configs in a completion
mode or an agent
mode. You set the mode
for a particular AI Config when you create it in the LaunchDarkly UI.
Customize AI Configs in completion mode
completion
mode means that each variation in your AI Config includes a single set of roles and messages that you use to prompt your generative AI model.
In completion
mode, use config()
to customize the AI Config. The config()
function takes an AI Config key, a context, a fallback value, and optional variables to use in the customization. It performs the evaluation, then returns the customized messages and model along with a tracker instance for recording metrics. If it cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. For example, you might use an empty, disabled LDAIConfig
as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.
Here’s how:
Customize AI Configs in agent mode
agent
mode means that each variation in your AI Config includes a set of instructions, which enable multi-step workflows.
In agent
mode, use agent()
or agents()
to customize the AI Config. The agent()
function customizes a single AI Config agent, while the agents()
function customizes an array of agent configurations.
Customization requires an AI Config key, fallback value, and optional variables to use in the customization, for each agent. Additionally, customization requires a context. Both functions perform the evaluation and then return the customized instructions for each AI Config, along with a tracker instance for recording metrics. If the function cannot perform the evaluation or LaunchDarkly is unreachable, it returns the fallback value. For example, you might use an empty, disabled LDAIAgentConfig
as a fallback value, or a fully configured default. Either way, you should make sure to check for this case and handle it appropriately in your application.
Here’s how:
To learn more, read Customizing AI Configs.
Call provider, record metrics from AI model generation
Finally, make a request to your generative AI provider and record metrics from your AI model generation.
If your AI Config uses completion
mode, the Node.js (server-side) AI SDK provides several options for this:
- If you are working with OpenAI or Bedrock Converse, use the
trackOpenAIMetrics
ortrackBedrockConverseMetrics
functions, respectively, to record metrics. These functions take the result of your generative operation as a parameter. - If you are working with Vercel, use either of the
trackVercelAISDKGenerateTextMetrics
ortrackVercelAISDKStreamTextMetrics
functions to record metrics. These functions take the result of a generative operation from any provider supported by the Vercel AI SDK as a parameter. - If you are using a generative AI provider or framework for which the SDK does not provide a convenience function, use the SDK’s other
track*
functions to record metrics manually.
If your AI Config uses agent
mode, you can access the instructions
returned from the agent()
call to send to your AI model. Use the tracker
returned in this call to record metrics.
Here’s how:
Here’s how to make a request using the Vercel AI SDK’s generateText
or streamText
, and record the metrics:
You can also use the SDK’s other track*
functions to record these metrics manually. You may need to do this if you are using a model for which the SDK does not provide a convenience track[Model]
function, and you are not using the Vercel AI SDK. The track[Model]
functions are expecting a response, so you may also need to do this if your application requires streaming.
In completion
mode, make sure to call config()
each time you generate content from your AI model:
To learn more, read Tracking AI metrics.
Supported features
This SDK supports the following features: