Databricks native Experimentation

1. Choosing an authentication method
2. Creating credentials in Databricks
3. Granting Unity Catalog permissions
4. Gathering your Databricks catalog and warehouse details
5. Configuring the LaunchDarkly integration
6. Verifying the connection

​

Prerequisites

• The Databricks Data Export integration set up in LaunchDarkly for the project and environment you use. Only environments with a Databricks Data Export destination are available when you create this integration.
• A Databricks workspace that uses Unity Catalog.
• A Databricks user who can create a service principal and grant Unity Catalog privileges. The service principal that LaunchDarkly uses does not need admin rights. To learn which privileges it needs, read Step 3.
• Catalog and schema names for experiment results, and optional metrics catalog and metrics schema names.
• An active Databricks SQL warehouse.
• A LaunchDarkly role of Owner or Admin, or a custom role that allows the following actions:
  • add and edit integrations
  • add destinations
  • add metric data sources
• Event data in your Databricks database that you want to use for Experimentation.
• An allow list in Databricks that includes the following outbound IP addresses:

​

Step 1: Choose an authentication method

• OAuth M2M: LaunchDarkly authenticates with the client ID and client secret of a Databricks service principal. This is the default method when the authentication method control appears in the configuration form.
• Personal access token: LaunchDarkly authenticates with a Databricks personal access token (PAT).

​

Step 2: Create credentials in Databricks

curl -X POST https://DATABRICKS_HOSTNAME/api/2.0/token/create \
  -H "Authorization: Bearer EXISTING_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "comment": "New PAT",
    "lifetime_seconds": LIFETIME_SECONDS
  }'

​

Step 3: Grant Unity Catalog permissions

• If the schema already exists: USE CATALOG on the catalog, and USE SCHEMA, CREATE TABLE, MODIFY, and SELECT on the schema.
• If you want LaunchDarkly to create the schema: USE CATALOG and CREATE SCHEMA on the catalog. The service principal owns the schema it creates and needs no schema-level grants.

• If you specify a Metrics catalog and Metrics schema: USE CATALOG on the catalog, and USE SCHEMA and SELECT on the schema.
• If you specify a Metrics catalog without a Metrics schema: USE CATALOG, USE SCHEMA, and SELECT on the catalog. This allows LaunchDarkly to read every schema in it.
• If a data source queries a table in any other catalog or schema, grant USE CATALOG, USE SCHEMA, and SELECT for that location too.

-- Experiment results schema that already exists
GRANT USE CATALOG ON CATALOG results_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, CREATE TABLE, MODIFY, SELECT ON SCHEMA results_catalog.results_schema TO `SERVICE_PRINCIPAL`;

-- Or, to let LaunchDarkly create the experiment results schema
GRANT USE CATALOG, CREATE SCHEMA ON CATALOG results_catalog TO `SERVICE_PRINCIPAL`;

-- Metrics catalog and schema (read only)
GRANT USE CATALOG ON CATALOG metrics_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, SELECT ON SCHEMA metrics_catalog.metrics_schema TO `SERVICE_PRINCIPAL`;

-- Or, to read every schema in the metrics catalog
GRANT USE CATALOG, USE SCHEMA, SELECT ON CATALOG metrics_catalog TO `SERVICE_PRINCIPAL`;

-- Data Export catalog and schema (read only)
-- Only if Experimentation uses a different service principal than Data Export
GRANT USE CATALOG ON CATALOG export_catalog TO `SERVICE_PRINCIPAL`;
GRANT USE SCHEMA, SELECT ON SCHEMA export_catalog.export_schema TO `SERVICE_PRINCIPAL`;

​

Step 4: Gather your Databricks catalog and warehouse details

1. Navigate to SQL > SQL Warehouses in Databricks.
2. Select the warehouse you plan to use.
3. Select the Connection details tab.
4. Copy and save the HTTP path.

• Catalog: The Unity Catalog catalog where LaunchDarkly writes experiment results.
• Schema: The schema within that catalog where LaunchDarkly writes experiment results.
• Metrics catalog (optional): The catalog that holds the data your metrics query, if it differs from the catalog above.
• Metrics schema (optional): The schema that holds the data your metrics query, if it differs from the schema above.

​

Step 5: Configure the LaunchDarkly integration

1. Navigate to the Integrations page.
2. Search for and select Databricks Native Experimentation.
3. Click Add integration. The configuration page appears.

4. Enter an integration Name.
5. Select the LaunchDarkly Project and environment for this integration. Only environments with a Databricks Data Export destination are available. You cannot change this after you save the integration.
6. Confirm the Workspace URL. LaunchDarkly fills this from the Databricks Data Export destination for the environment you selected. You cannot edit it.
7. Confirm or update the HTTP path. LaunchDarkly copies this from the Data Export destination. You can change it to query a different SQL warehouse in the same workspace. You cannot change it after you save.
8. Select an Authentication method. LaunchDarkly selects OAuth M2M by default. Click Personal access token to authenticate with a token instead. You cannot change the method after you save.
9. Enter the credentials for the method you selected:
   • OAuth M2M: The Client ID and Client secret you saved in Step 2.
   • Personal access token: The PAT you saved in Step 2.
10. Enter your Databricks Catalog.
11. Enter your Databricks Schema.
12. (Optional) Enter your Databricks Metrics catalog.
13. (Optional) Enter your Databricks Metrics schema.
14. After reading the Integration Terms and Conditions, check the I have read and agree to the Integration Terms and Conditions checkbox.
15. Click Test connection and save.

​

Step 6: Verify the connection

​

Edit an existing integration

​

Next steps

• Creating experiments using warehouse native metrics
• Analyzing experiments