This guide provides examples of metrics you can use with LaunchDarkly features such as Experimentation and guarded rollouts.
Metrics measure audience behaviors in your app or product and how those behaviors are affected by different flag variations. You can use metrics to track all kinds of things, from how often end users access a URL to how long that URL takes to load a page. How you set up a metric varies depending on what you want to measure. The examples included in this guide will help you understand how to configure metrics to meet your business needs.
To learn more about creating metrics, read Creating metrics.
This section includes the configuration options for common custom numeric metrics related to average purchase price per user. For all of the examples in this section, the randomization unit is “user.”
You can use this custom numeric metric to learn the average amount of money spent by users who made a purchase.
The metric configuration options include:
Event kind: Custom
Definition: Average event values per user, then compute the Average of those values where Higher is better
Units without events: Excluded
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This means that buyers spend an average of $9.
You can use this custom numeric metric to learn the average amount of money spent by all users, whether or not they made a purchase.
The metric configuration options include:
Event kind: Custom
Definition: Average event values per user, then compute the Average of those values where Higher is better
Units without events: Include units and set the value to 0
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This means that all users, including buyers and non-buyers, spend an average of $6.75.
This section includes the configuration options for a custom numeric metric related to percentile purchase price per user. For the example in this section, the randomization unit is “user.”
The default metric analysis method is “Average.” The use of percentile analysis methods with LaunchDarkly experiments is in beta. If you use a metric with a percentile analysis method in an experiment with a large audience, the experiment results tab may take longer to load, or the results tab may time out and display an error message. Percentile analysis methods are also not compatible with CUPED adjustments.
In this example, you are only interested in learning about your typical spenders and want to exclude the few customers who spend significantly more than most and skew your data upward. This metric calculates the amount of money that 90% of your customer base spends less than.
This custom numeric metric only includes users who made a purchase. When using a percentile analysis method, LaunchDarkly automatically excludes units without events.
The metric configuration options include:
Event kind: Custom
Definition: Average event values per user, then compute the P90 of those values where Higher is better
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This means that 90% of all users that make a purchase spend less than $85.
This section includes the configuration options for common custom numeric metrics related to average revenue per user. For all of the examples in this section, the randomization unit is “user.”
This custom numeric metric sums purchase amounts per user who bought something, instead of calculating average amounts per purchase. This lets you find out about the purchasing behavior of users as a whole, ignoring whether the money they spent was all at once or spread between multiple purchases.
The metric configuration options include:
Event kind: Custom
Definition: Sum event values per user, then compute the Average of those values where Higher is better
Units without events: Excluded
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This custom numeric metric sums purchase amounts per user, instead of calculating average amounts per purchase. This lets you find out about the purchasing behavior of users as a whole, ignoring whether the money they spent was all at once or spread between multiple purchases. This includes users who didn’t purchase anything.
The metric configuration options include:
Event kind: Custom
Definition: Sum event values per user, then compute the Average of those values where Higher is better
Units without events: Include units and set the value to 0
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This section includes the configuration options for common custom conversion binary metrics related to average number of purchases. For the example in this section, the randomization unit is “user.”
You can use this custom conversion binary metric to learn the average number of purchases for users who made a purchase.
The metric configuration options include:
Event kind: Custom
Definition: Count events per user, then compute the Average of those values where Higher is better
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This section includes the configuration options for common custom numeric metrics related to latency time. For the example in this section, the randomization unit is “request.”
The default metric analysis method is “Average.” The use of percentile analysis methods with LaunchDarkly experiments is in beta. If you use a metric with a percentile analysis method in an experiment with a large audience, the experiment results tab may take longer to load, or the results tab may time out and display an error message. Percentile analysis methods are also not compatible with CUPED adjustments.
This custom numeric metric calculates latency, excluding the slowest 1% of requests that may skew the data. When using a percentile analysis method, LaunchDarkly automatically excludes units without events.
The metric configuration options include:
Event kind: Custom
Definition: Average event values per user, then compute the P99 of those values where Higher is better
Here is what the metric setup looks like:
Here is an example of how the metric calculates its results:
This means that 99% of requests, on average, have a latency of less than 70ms.
The guide explained how to set up metrics to help answer common questions about your app’s revenue and latency performance. For more examples of experiments you can run, read Example experiments.