ExperimentationExperimentation

Quickstart for Experimentation

Overview

This topic explains how to complete the quickstart for LaunchDarkly’s Experimentation product.

Follow the instructions below to complete the Experimentation quickstart and create your first experiment.

Important concepts

As you progress through the Experimentation quickstart, you may encounter terminology that’s new to you. Here’s a brief explanation of what some of these terms mean. If you want to go into more detail about anything in this section, read Experimentation and metric events.

Events and metrics

Two distinct, interconnected resources in LaunchDarkly are “events” and “metrics.”

  • Events are raw data that represent specific actions, such as a button click or a page view. They are instrumented in code, and used to create metrics.
  • Metrics define how event data should be aggregated and interpreted so you can measure the effect of a change. They are created in the LaunchDarkly user interface (UI).

For example, an event can log each time a user clicks “Buy,” while the metric measures the average conversion rate of this action for your website’s visitors.

Best practices for identifying useful data when designing experiments

Start by identifying actions relevant to most experiments, such as conversion rate or page load time, and create events representing those in LaunchDarkly.

As you build experiments, gradually add additional more specific events and corresponding metrics. When you define metrics in LaunchDarkly, here some of the things to keep in mind:

  • Consider the unit you’re using. To use a given metric in an experiment, the metric’s unit needs to be the same as the experiment’s randomization unit. If you’re familiar with other aspects of LaunchDarkly, it will help you to know that the experiment’s unit is synonymous with its context kind.
  • You can often derive additional helpful metrics from a single conversion event. Depending on the experiment’s hypothesis, it might be helpful to measure the event occurrence (the portion of users who converted at least once), the event count (the average number of conversion events per user), or the average value associated with the given conversion event.

Using the quickstart

The quickstart takes you through the core actions to create an experiment in a controlled, abbreviated workflow. You will choose and initialize a LaunchDarkly SDK, create a flag and metrics, and instrument events to monitor when your experiment gets interaction. These are the core steps to begin an experiment.

Here’s how to begin the Experimentation quickstart.

  1. Log in to LaunchDarkly.
  2. In the lower left corner, click the question mark icon. A menu appears.
  3. In the “Developer” section, choose Quickstart. The quickstart appears.
  4. Click Experiments to expand the experimentation quickstart steps.

Step 1: Connect your app

To begin, you must install a LaunchDarkly SDK locally. Here’s how:

  1. Click to expand Step 1: Connect your app.
  2. Choose an SDK from the dropdown menu. Its installation instructions appear.
  3. Install the SDK in your app:
1<dependency>
2<groupId>com.launchdarkly</groupId>
3<artifactId>launchdarkly-java-server-sdk</artifactId>
4<version>7.0.0</version>
5</dependency>
  1. Initialize the SDK:
1import com.launchdarkly.sdk.*;
2import com.launchdarkly.sdk.server.*;
3
4// A "context" is a data object representing users, devices, organizations, and
5// other entities. You'll need this later, but you can ignore it for now.
6final LDContext context = LDContext.builder("example-context-key")
7.name("Sandy")
8.build();
9
10LDConfig config = new LDConfig.Builder().build();
11
12// Set your LaunchDarkly SDK key.
13// This is inlined as example only for onboarding.
14// Never hardcode your SDK key in production.
15final LDClient client = new LDClient("sdk-378e1611-aa33-4fa2-b477-9279c59a6c59", config);
16
17if (client.isInitialized()) {
18System.out.println("SDK successfully initialized!");
19}

Step 2: Create a flag

Next, create a flag to track your experiment. Here’s how:

  1. Click to expand Step 2: Create a flag. A flag create menu appears.
  2. Give your flag a human-readable Name and click Create flag.
  3. Test the flag by wrapping a feature in your app in the flag code. Here’s how:
1// Set up the evaluation context.
2final LDContext context = LDContext.builder("example-context-key")
3  .name("Sandy")
4  .build();
5
6// Evaluate the feature flag for this context.
7boolean flagValue = client.boolVariation('my-cool-flag', context, false);
8
9if (flagValue) {
10
11  // TODO: Put your feature here
12
13} else {
14
15  // TODO: Put your fallback behavior here
16
17}
  1. Run your application to verify that it works as you expect.
  2. Navigate to the flag’s Targeting tab and turn the flag on. To learn more, read Target with flags.
  3. Return to the tab in your browser with the Experimentation quickstart.

Step 3: Instrument events

Events let you track the raw data that matters for your experiment, like clicks, pageviews, and custom events.

For this quickstart, we recommend you use custom events. You can send them using the SDK’s track method. Here’s how:

  1. Click to expand Step 3: Instrument events.
  2. Use the code below to instrument events for your experiment.
1client.track("event-key-123abc", context);

Step 4: Create a metric

Metrics measure the outcome of an experiment. Conversion metrics compare two different versions against each other to see which produces a better result.

To learn more about how to create metrics, read Creating and managing metrics.

  1. Click to expand Step 4: Create a metric.
  2. Click Create metric. The metric creation menu appears.
  3. Enter the Event key you instrumented in the previous step.
  4. Choose how you want to measure the conversion event: by Count, Occurrence, or Value / Size.
  5. Give your metric a human-readable Name.
  6. Click Create metric.

Step 5: Create an experiment

Now it’s time to create your experiment.

Click to expand Step 5: Create an experiment, then click Create experiment. The experiment builder appears, and now you’re ready to go!

There are many ways to customize your experiment to track exactly the information you need. Here are some recommendations to create the simplest possible experiment:

  1. Add a clear hypothesis to the experiment.
  2. Choose a feature change experiment to get started as fast as possible
  3. When you choose a randomization unit, try “users.” Users are the most common randomization unit.
  4. Set up a 50/50 traffic split to see accurate, significant results quickly.

To learn more, read Experimentation.

Built with