This guide provides examples of metrics you can use with LaunchDarkly features such as Experimentation and guarded rollouts.Metrics measure audience behaviors in your app or product and how those behaviors are affected by different flag variations. You can use metrics to track all kinds of things, from how often end users access a URL to how long that URL takes to load a page. How you set up a metric varies depending on what you want to measure. The examples included in this guide will help you understand how to configure metrics to meet your business needs.To learn more about creating metrics, read Creating metrics.
This section includes the configuration options for common custom numeric metrics related to average purchase price per user. For all of the examples in this section, the randomization unit is “user.”
Average purchase price per user: only users who made a purchase
Expand Average purchase price per user: only users who made a purchase
You can use this custom numeric metric to learn the average amount of money spent by users who made a purchase.The metric configuration options include:Event kind: CustomDefinition: Average event values per user, then compute the Average of those values where Higher is betterUnits without events: ExcludedHere is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: $5, $15
User B: $10
User C: $7
User D: $0 (no purchases)
The metric calculates the average for each user, excluding those who didn’t purchase anything:
User A: $10
User B: $10
User C: $7
The metric then calculates the average purchase price per user:
You can use this custom numeric metric to learn the average amount of money spent by all users, whether or not they made a purchase.The metric configuration options include:Event kind: CustomDefinition: Average event values per user, then compute the Average of those values where Higher is betterUnits without events: Include units and set the value to 0Here is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: $5, $15
User B: $10
User C: $7
User D: $0 (no purchases)
The metric calculates the average for each user, including those who didn’t purchase anything:
User A: $10
User B: $10
User C: $7
User D: $0
The metric then calculates the average purchase price per user:
Result: $6.75
This means that all users, including buyers and non-buyers, spend an average of $6.75.
This section includes the configuration options for a custom numeric metric related to percentile purchase price per user. For the example in this section, the randomization unit is “user.”
Expand Percentile purchase price per user
Percentile analysis is not available for all experimentsYou can use percentile analysis with frequentist experiments that use sequential testing.You cannot use percentile analysis with Bayesian experiments or with frequentist experiments that don’t use sequential testing.
In this example, you are only interested in learning about your typical spenders and want to exclude the few customers who spend significantly more than most and skew your data upward. This metric calculates the amount of money that 90% of your customer base spends less than.This custom numeric metric only includes users who made a purchase. When using a percentile analysis method, LaunchDarkly automatically excludes units without events.The metric configuration options include:Event kind: CustomDefinition: Average event values per user, then compute the P90 of those values where Higher is betterHere is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: $7
User B: $9
User C: $5, $15
User D: $12
User E: $21
User F: $20, $40
User G: $45
User H: $78
User I: $80
User J: $85
User K: $550
User L: No purchases
The metric calculates the average for each user that made a purchase:
User A: $7 (0th percentile)
User B: $9 (10th percentile)
User C: $10 (20th percentile)
User D: $12 (30th percentile)
User E: $21 (40th percentile)
User F: $30 (50th percentile)
User G: $45 (60th percentile)
User H: $78 (70th percentile)
User I: $80 (80th percentile)
User J: $85 (90th percentile)
User k: $550 (100th percentile)
User L: Excluded from calculation
The metric then calculates the average for the 90th percentile:
Result: $85
This means that 90% of all users that make a purchase spend less than $85.
This section includes the configuration options for common custom numeric metrics related to average revenue per user. For all of the examples in this section, the randomization unit is “user.”
Average revenue per user: only users who generated revenue
Expand Average revenue per user: only users who generated revenue
This custom numeric metric sums purchase amounts per user who bought something, instead of calculating average amounts per purchase. This lets you find out about the purchasing behavior of users as a whole, ignoring whether the money they spent was all at once or spread between multiple purchases.The metric configuration options include:Event kind: CustomDefinition: Sum event values per user, then compute the Average of those values where Higher is betterUnits without events: ExcludedHere is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: $100, $300
User B: $10
User C: $7
User D: $0 (no purchases)
The metric calculates the sum for each user, excluding those who didn’t purchase anything:
This custom numeric metric sums purchase amounts per user, instead of calculating average amounts per purchase. This lets you find out about the purchasing behavior of users as a whole, ignoring whether the money they spent was all at once or spread between multiple purchases. This includes users who didn’t purchase anything.The metric configuration options include:Event kind: CustomDefinition: Sum event values per user, then compute the Average of those values where Higher is betterUnits without events: Include units and set the value to 0Here is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: $100, $300
User B: $10
User C: $7
User D: $0 (no purchases)
The metric calculates the sum for each user, including those who didn’t purchase anything:
This section includes the configuration options for common custom conversion binary metrics related to average number of purchases. For the example in this section, the randomization unit is “user.”
Average number of purchases: per user who purchased
Expand Average number of purchases: per user who purchased
You can use this custom conversion binary metric to learn the average number of purchases for users who made a purchase.The metric configuration options include:Event kind: CustomDefinition: Count events per user, then compute the Average of those values where Higher is betterHere is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Step 3
Users made the following purchases:
User A: 1 purchase, 1 purchase
User B: 1 purchase
User C: 1 purchase
User D: 0 purchases
The metric calculates the sum for each user, excluding those who didn’t purchase anything:
This section includes the configuration options for common custom numeric metrics related to latency time. For the example in this section, the randomization unit is “request.”
Expand Latency excluding the slowest 1% of results
Percentile analysis is not available for all experimentsYou can use percentile analysis with frequentist experiments that use sequential testing.You cannot use percentile analysis with Bayesian experiments or with frequentist experiments that don’t use sequential testing.
This custom numeric metric calculates latency, excluding the slowest 1% of requests that may skew the data. When using a percentile analysis method, LaunchDarkly automatically excludes units without events.The metric configuration options include:Event kind: CustomDefinition: Average event values per user, then compute the P99 of those values where Higher is betterHere is what the metric setup looks like:
The metric creation dialog.
Here is an example of how the metric calculates its results:
Step 1
Step 2
Requests had the following latency:
Request A: 20ms
Request B: 10ms
Request C: 70ms
Request D: 30ms
The metric then calculates the 99th percentile:
Result: 70ms
This means that 99% of requests, on average, have a latency of less than 70ms.
The guide explained how to set up metrics to help answer common questions about your app’s revenue and latency performance. For more examples of experiments you can run, read Example experiments.
Want to know more? Start a trial.Your 14-day trial begins as soon as you sign up. Get started in minutes using the in-app Quickstart. You’ll discover how easy it is to release, monitor, and optimize your software.
