> ## Documentation Index
> Fetch the complete documentation index at: https://launchdarkly.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Databricks Data Export
**This integration is not available in federal environments**
To learn more, read LaunchDarkly in federal environments.
**Data Export is an add-on feature**
Data Export is available as an add-on feature to select plans. To learn more, [read about our pricing](https://launchdarkly.com/pricing/). To add Data Export to your plan, [contact Sales](https://launchdarkly.com/contact-sales/).
This topic explains how to create and test a Databricks destination for Data Export. Databricks is a cloud-based data processing and analysis platform that lets you work with large sets of data. By exporting your LaunchDarkly experiment data to the same Databricks warehouse as your other data, you can build custom reports in Databricks to answer product behavior questions.
You can also run experiments using warehouse native metrics. To learn more, read [Creating experiments using warehouse native metrics](/docs/home/warehouse-native/creating).
To set up Databricks Data Export, you must:
* [Create a service principal](#create-a-service-principal)
* [Choose an authentication method](#choose-an-authentication-method)
* [Create a SQL warehouse](#create-a-sql-warehouse)
* [Give catalog and schema permissions](#give-catalog-and-schema-permissions)
* [Set up the integration in LaunchDarkly](#set-up-the-integration)
## Prerequisites
To configure the Databricks Data Export integration, you must have the following prerequisites:
* You must have a [LaunchDarkly role](/docs/home/account/roles) that lets you [add and edit integrations](/docs/home/account/roles/role-actions#integration-actions) and [add destinations](/docs/home/account/roles/role-actions#destination-actions)
- You need to allow the LaunchDarkly data transfer service's static IP address: `35.192.85.117`
* You need to allow the LaunchDarkly data transfer service's static IP address: `35.241.154.37`
**One Data Export destination per environment**
You can configure only one Data Export destination per LaunchDarkly environment, regardless of destination kind. If you need more than one Data Export destination for the same data, configure them in different environments.
## Create a service principal
We recommended that you use a Databricks [Service principal](https://docs.databricks.com/aws/en/admin/users-groups/service-principals) to provide LaunchDarkly service access to your warehouse.
To create a new service principal, follow the Databricks instructions to [Add service principals to your account](https://docs.databricks.com/aws/en/admin/users-groups/manage-service-principals#-add-service-principals-to-your-account).
Ensure that the principal has **Workspace access** and **Databricks SQL access** entitlements. To learn how, read the Databricks documentation about [Access entitlements](https://docs.databricks.com/aws/en/security/auth/entitlements).
## Choose an authentication method
LaunchDarkly supports two methods for authenticating the Databricks Data Export integration:
* **OAuth M2M**: LaunchDarkly authenticates with the client ID and client secret of a Databricks service principal. This is the default method when the authentication method control appears in the configuration form.
* **Personal access token**: LaunchDarkly authenticates with a Databricks personal access token (PAT).
You select the authentication method when you create the integration. You cannot change it after you save the configuration. To use different OAuth credentials or to switch methods, delete the integration and create a new one.
If the "Authentication method" option does not appear in the configuration form, OAuth M2M is not enabled for your account. Use a personal access token instead.
### Create OAuth M2M credentials
LaunchDarkly connects to your warehouse as the service principal using OAuth machine-to-machine (M2M) authentication. Generate an OAuth secret for the service principal, then provide its client ID and secret to LaunchDarkly. To learn more, read the Databricks documentation about [OAuth machine-to-machine (M2M) authentication](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m).
To generate the credentials:
1. In the Databricks account console, click **Settings**, then open **Identity and access** and find "Service principals."
2. Select the service principal you created, then copy its **Application ID**. This is your client ID.
3. Open the service principal's **Secrets** tab, then click **Generate secret**.
4. Set a **Lifetime (days)** for the secret, then click **Generate**.
5. Copy the **Secret** and save it in a secure location. Databricks displays the secret only once. This is your client secret.
You will provide the client ID and secret later when [setting up the integration](#set-up-the-integration) in LaunchDarkly. LaunchDarkly uses the client secret to connect to Databricks but does not store it after setup.
### Create a personal access token
Use this procedure if you select **Personal access token** when you set up the integration. First create a workspace token in Databricks. Then use that token to create the PAT that LaunchDarkly uses. Finally, grant your service principal permission to use personal access tokens.
To create a workspace token:
1. Navigate to **Workspace settings** > **Developer**.
2. Click **Manage**.
3. Click **Generate new token**.
4. Copy and save the token. You use this token as the bearer token in the next request.
Create the PAT that LaunchDarkly uses by sending the following request. Replace `DATABRICKS_HOSTNAME` with your workspace hostname, `EXISTING_TOKEN` with the token from the previous steps, and `LIFETIME_SECONDS` with the token lifetime in seconds:
```bash title="API access token" lines wrap theme={null}
curl -X POST https://DATABRICKS_HOSTNAME/api/2.0/token/create \
-H "Authorization: Bearer EXISTING_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"comment": "New PAT",
"lifetime_seconds": LIFETIME_SECONDS
}'
```
Copy and save the PAT from the response. Provide this PAT to LaunchDarkly when you [set up the integration](#set-up-the-integration).
Then grant the service principal permission to use personal access tokens:
1. Navigate to **Workspace settings** > **Workspace admin** > **Advanced** > **Personal access tokens** > **Permission settings**.
2. Add the service principal you created.
3. Assign the **Can use** permission.
## Create a SQL warehouse
Now, follow the Databricks instructions to [Create a new SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create).
Under [permissions](https://docs.databricks.com/aws/en/compute/sql-warehouse/create#manage-a-sql-warehouse), assign the **Can use** permission to the service principal you created above.
Under **Connection details**, find and save the **Server hostname** and **HTTP path** of the warehouse. You will need these when setting up the integration in LaunchDarkly.
## Give catalog and schema permissions
Then, give the service principal you created the appropriate permissions for your [destination catalog](https://docs.databricks.com/aws/en/catalogs/) and [destination schema](https://docs.databricks.com/aws/en/schemas/).
For your destination catalog, give the service principal the `USE CATALOG` permission.
For your destination schema, give the service principal the following grants:
* `USE SCHEMA`
* `APPLY TAG`
* `MODIFY`
* `READ VOLUME`
* `SELECT`
* `WRITE VOLUME`
* `CREATE MATERIALIZED VIEW`
* `CREATE TABLE`
* `CREATE VOLUME`
Databricks selects "Unity Catalog" for metastore type by default. If your workspace uses the [legacy Hive metastore](https://docs.databricks.com/aws/en/data-governance/unity-catalog/hive-metastore), select it instead and provide the required S3 bucket details and access keys.
## Set up the integration
To configure the integration in LaunchDarkly:
1. Click the **gear** icon in the left sidebar to view Organization settings.
2. Click **Integrations** and find "Databricks Data Export."
3. Click **Add integration**. The Databricks Data Export configuration page appears.
4. Give the integration a human-readable **Name**.
5. Choose a **Project and environment** to export data from. Environments that already have a Data Export destination are unavailable.
6. Enter the **Server hostname** you saved when you [created a SQL warehouse](#create-a-sql-warehouse). You can enter a bare hostname or prefix it with `https://`, for example `dbc-abcd1234-5678.cloud.databricks.com` or `https://dbc-abcd1234-5678.cloud.databricks.com`. Do not use `http://`. The value must be a hostname, with no path and no extra whitespace.
7. Enter the **HTTP path** you saved when you created a SQL warehouse. The path starts with `/sql/`, for example `/sql/1.0/warehouses/abc123`.
8. Enter the **Catalog** and **Schema** to export data to. Catalog and schema names can contain only letters, numbers, and underscores. Hyphens are not allowed.
9. Select an **Authentication method**. LaunchDarkly selects **OAuth M2M** by default. Click **Personal access token** to authenticate with a token instead.
10. Enter the credentials for the method you selected:
* **OAuth M2M**: The **Client ID** and **Client secret** you [created in a previous step](#create-oauth-m2m-credentials). LaunchDarkly uses these credentials to connect to Databricks but does not store the client secret after setup.
* **Personal access token**: The PAT you [created in a previous step](#create-a-personal-access-token).
11. Select a **Metastore** of "Unity Catalog" or "Hive." LaunchDarkly selects "Unity Catalog" by default.
12. Enter the staging bucket details if you selected the "Hive" metastore: the **S3 bucket name**, the **S3 bucket region**, the **Bucket access ID**, and the **Bucket secret key**. LaunchDarkly uses the bucket secret key to connect but does not store it after setup.
13. Read the Integration Terms and Conditions, then select the **I have read and agree to the Integration Terms and Conditions** checkbox.
14. Click **Test connection and save**. LaunchDarkly tests the connection and, if it succeeds, activates the destination and starts the initial transfer.
Your Databricks Data Export destination is now configured.
To learn about the different event kinds for Databricks Data Export destinations, read [Warehouse Data Export schema reference](/docs/integrations/data-export/warehouse-schema-reference).
## View an existing destination
The destination setup is visible on the edit page so that you can confirm which host, HTTP path, catalog, schema, metastore, and client ID a destination uses. However, all fields on the edit page are read-only.
LaunchDarkly never stores the secrets you provide at setup, so it cannot display them again. The **Client secret** field and the **Bucket secret key** field for a Hive metastore show a masked placeholder to indicate that you provided a secret when you created the destination.
To view an existing Databricks Data Export destination:
1. Click the **gear** icon in the left sidebar to view Organization settings.
2. Click **Integrations** and find "Databricks Data Export."
3. Find the destination you want to view and click **Edit**. The Databricks Data Export configuration page appears with the destination's settings.
To verify that the destination still works, click **Send event** to send a test event from LaunchDarkly to your warehouse. To change any connection value, delete the destination and configure a new one.
## Health checks
If your warehouse Data Export integration develops connection problems or other errors, the integration displays a health check warning. To learn more, read [Warehouse health checks](/docs/home/warehouse-native/warehouse-health-checks).