Databricks Lakehouse

🐕‍🦺 Setup guide

Zenskar's Databricks data-source connector syncs data with Delta Lake on Databricks Lakehouse. Each stream is written to its own Delta table.

🚧

Note

You must use Unity Catalog to use this connector.

⚙️ Step 1: set up data source and type

1. Log into your Zenskar account.
2. In the left navigation bar, click Metering > Data Sources.
3. In the top-right corner, click + ADD DATA SOURCE.
4. In the Set Up Source section of the Add New Data Source page, enter a name for the Databricks data source connection.
5. Select Databricks from the Source Type drop-down menu.

⚙️ Step 2: configure data source

In the Source Config section of the Add New Data Source page, fill in the following details:

• Databricks Access Token: refer the Databricks authentication section.
• Workspace Hostname: the hostname of the Databricks workspace. Refer the Section Retrieve Databricks workspace details section.

A workspace is a Databricks deployment in a cloud service account. It provides a unified environment for working with Databricks assets for a specified set of users.

• Port Number: refer the Retrieve Databricks workspace details section.
• HTTP Path: refer the Retrieve Databricks workspace details section.

Retrieve Databricks workspace details

1. Open the workspace console.
2. Open your SQL warehouse:

3. Open the Connection Details tab:

Databricks authentication

Generate access token

1. Open your workspace console.
2. Click on the named drop-down menu at the top-right corner, and click on Settings option.
3. Navigate to User Settings > Developer, and click on Generate new token, as shown below.

4. Enter a description for the token and its lifetime. You can leave blank Lifetime (days) blank for a permanent token:

Supported sync modes