Feature flags and AI ConfigsAI Configs

Monitor AI Configs

Overview

This topic explains how to monitor the performance of your AI Configs. Performance metrics for AI Configs are available in the LaunchDarkly user interface if you track AI metrics in your SDK.

Monitor an AI Config

To monitor the performance of an AI Config:

  1. Navigate to the detail page for the AI Config. Select the Monitoring tab.
  2. At the top of the tab, use the controls to specify the monitoring data you want to view:
  • Use the Environment dropdown to select the environment you want to monitor. Performance metrics are specific to each environment.
  • Select a date range.
  • Use the Charts dropdown to select which set of charts to view.
  1. Review charts of the available monitoring data. The data in each chart applies to the selected environment and is split out by variation:
  • The Tokens charts include average input and output tokens used by the AI Config in this environment, by variation.
  • The Satisfaction chart is the percentage of “thumbs up” ratings provided by end users who have encountered the AI Config in this environment, by variation.
  • The Generations is the average number of successful generations completed using the AI Config in this environment, by variation.
  • The Time to generate is the average duration per generation. In other words, this is the total duration of calls to your LLM provider divided by the number of total generations using the AI Config in this environment, by variation.
  • Error rate is the percentage of errors out of the total number of generations attempted for the AI Config in this environment, by variation.
  • Time to first token is the mean time it takes to generate the initial token.
  1. Review the table of available monitoring data. The table includes the data displayed in the charts as well as additional data. For example, the table includes both the average and the total input and output tokens.
  2. (Optional) In the table, select the variations or versions for which you want to view data. The charts update based on your selection.
  3. (Optional) Click Export data as CSV to download a CSV file for further analysis.

In order for the data on the Monitoring tab to appear, you must record the metrics in your application, using a track call from any of the LaunchDarkly AI SDKs. To learn more, read Tracking AI metrics.

The data on the Monitoring tab updates approximately each minute. For each metric, the results are broken out by AI Config variation. Metrics with no results display “No data.” If there is no data for a particular variation, that variation is not included in the total displayed at the top of the metric card.