Filtering experiment results

This topic explains how to use attribute filters to narrow experiment results by attribute. Attribute filters allow you to analyze your experiment results by certain cross-sections of your experiment audience using context attributes. For example, if you wanted to view only results from users within Canada, you might narrow your results by users with a “Country” attribute value of Canada.

​

Add attribute filters

To filter results by attribute, you can add up to 20 attributes in the “Advanced” section when you create an experiment. To learn how, read Creating experiments. You can then filter your results by the different values of one of those attributes. You can also add attributes to an existing experiment from its Results tab:

1. Navigate to your experiment’s Results tab.
2. Scroll to the results table and click Add.
3. Select Attributes. An “Add attributes” dialog appears, listing the context attributes you can add.

4. Select the attributes you want to filter by. You can also type the name of an attribute that LaunchDarkly has not observed yet.
5. Click Add.

You can now filter your experiment results by these attributes.

​

View results filtered by attribute

To view experiment results filtered by attribute:

1. Navigate to your experiment’s Results tab.
2. Above the results table, click All attributes. A menu of attributes appears.
3. Hover over an attribute from the menu. A list of attribute values appears.
4. Check the box next to any attribute value you want to narrow your results by.

The graph updates to display results only from contexts that have the attribute values you selected:

​

Statistical adjustments

LaunchDarkly does not apply covariate adjustment (CUPED), multiple comparison correction (MCC), or sequential testing to filtered attribute results, even if LaunchDarkly applied those adjustments to the unfiltered results. We recommend using filtered results for exploration and hypothesis generation rather than final decision-making.

​

Attribute value update timing

After you add attributes to an existing experiment, the time it takes for LaunchDarkly to update the attribute values depends on the metric type and age of the experiment iteration:

Metric type	Iteration age	Time to update
LaunchDarkly hosted	Less than one day	Approximately five minutes
LaunchDarkly hosted	1-60 days	Approximately one hour
LaunchDarkly hosted	More than 60 days	Approximately one day
Warehouse native	Any	After sync with external warehouse, usually 1-2 hours

​

Attribute eligibility

You can add any attribute from your context data to an experiment, either when you create the experiment or while it is running. If an attribute has no data yet, its results display as empty until LaunchDarkly records values for it.

​

Eligibility for older experiment iterations

Some older experiment iterations that use LaunchDarkly hosted metrics still use eligibility thresholds. For these iterations, the “Add attributes” dialog lists each attribute as eligible or ineligible based on how often LaunchDarkly observes it in your experiment audience. An attribute is eligible when its most common value:

• appears for at least 50 contexts in the experiment, and
• accounts for at least 5% of the experiment audience.

Attributes below these thresholds are ineligible. If the iteration is new or has low traffic, the dialog may display “No attributes available yet.” Attributes become eligible as the iteration collects more exposures.

​

Attribute value limits

For each attribute, LaunchDarkly displays results for up to the 20 most frequent values, ranked by the number of contexts having each value. All remaining values are combined into a single “(other)” group. If an attribute has 20 or fewer values, LaunchDarkly displays all of them and no “(other)” group appears. Limiting the number of values keeps results readable for attributes with many distinct values, and it reduces the risk of false positives from comparing many small segments. The limit also bounds the size of the queries that LaunchDarkly runs to calculate results. LaunchDarkly calculates results for the “(other)” group by combining the contexts from all remaining values into a single group, then calculating the metric directly over those contexts. The result is the same as if the “(other)” group were one attribute value. For mean and ratio metrics, the values you see always add up to your experiment totals. For percentile metrics, the “(other)” result is a percentile of the combined group. Percentile results never add up across values, because percentiles cannot be combined after they are calculated. This applies to both named values and to the “(other)” group. LaunchDarkly recalculates the top 20 values each time it updates results, over the full duration of the experiment iteration. If your data changes and a different value enters the top 20, the displayed values update the next time results update.