BigQuery

Overview

Google BigQuery is a fully managed, serverless data warehouse that enables scalable analysis over large datasets. In Zenskar, you can use BigQuery as a data source for billable metrics and dashboards.

Prerequisites

Enable BigQuery API in your GCP before connecting it to Zenskar. You can enable BigQuery API directly from your project too..
Create a service account.
Add the Service Account as a Member in your GCP account with the BigQuery User role.
Similarly, assign the BigQuery Data Editor role to the Service Account.

📖
We highly recommend that you create a Service Account exclusive to Zenskar for ease of permission and auditing. However, you can use a pre-existing Service Account that has the correct permissions.

Create and manage a Service Account key in Google Cloud Project

Service Account keys are used to authenticate as Google Service Accounts. Zenskar requires the Service Account keys to leverage the role-based permissions you granted to the Service Account in the previous section.
Create and manage a Service Account key. Ensure the following:
- Create your key in JSON format.
- Download the key immediately.
Google will not allow you to see the contents of the key once you navigate away.

⚠️
Delete the downloaded key file after setup for security.

Set up a BigQuery data-source connector via Zenskar dashboard

Log into your Zenskar dashboard.
In the left side bar, click Usage > Data Sources.
In the top-right corner, click + ADD DATA SOURCE.
On the Add New Data Source page, configure the BigQuery connector.
Click on the SAVE SOURCE button.

Connector configuration

General configuration

Field	Description	Required
Source Name	Enter a unique name for this data source.	Yes
Source Type	Select BigQuery from the dropdown menu.	Yes

Connector configuration

Field	Description	Required
Project ID	The ID of your Google Cloud project that contains the BigQuery datasets.	Yes
Dataset ID	The dataset you want to query.	Yes
Location	The geographic location of your BigQuery dataset (for example, `US` or `EU`).	Yes
Credentials JSON	Paste the contents of your Service Account Key (JSON format).	Yes

Data source access mode (read-only)

Zenskar queries data directly from BigQuery without syncing it to Zenskar’s data infrastructure. This option is:

Ideal for large databases (more than 30 GB)
Suitable for real-time data access
No sync waiting time

You will be able to browse and query tables from this BigQuery source in the Data Navigator while creating aggregates.

Set up a BigQuery data-source connector via API

Refer to the create data-source connector API reference.

💡
The connector_config object is the only part of the request that differs across connector types (such as BigQuery, Snowflake, or Redshift). The create data-source connector API reference provides a generic overview, while this document explains the BigQuery-specific structure of connector_config.

Request example

curl --location 'https://api.zenskar.com/datasources' \
  -H 'x-api-key: <your-api-key>' \
  -H 'apiversion: 20240301' \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "abc",
    "connector_type": "BigQuery",
    "destination": "BigQuery",
    "status": "active",
    "connector_config": {
      "project_id": "my-test-project-84030",
      "dataset_id": "abc123",
      "location": "US",
      "credentials_json": "{ \"type\": \"service_account\", \"project_id\": \"my-test-project-84030\", \"private_key_id\": \"0533306f55057\", \"private_key\": \"-----BEGIN PRIVATE KEY-----\\t/YgHKB4tH\\n-----END PRIVATE KEY-----\\n\", \"client_email\": \"[email protected]\", \"token_uri\": \"https://oauth2.googleapis.com/token\" }"
    },
    "source_definition_id": "bfd1ddf8-ae8a-4620-b1d7-55597d2ba08c",
    "remote_conn": true
  }'

BigQuery-specific connector configuration

The connector_config object contains configuration fields specific to the BigQuery data source.

Field	Description	Required
`project_id`	The unique ID of your Google Cloud project that contains the BigQuery datasets.	Yes
`dataset_id`	The dataset name (for example, `abc123`). Do not include the project ID prefix (such as `my-test-project-84030.abc123`). The project ID is already supplied separately.	Yes
`location`	The geographic location of your BigQuery dataset (for example, `US` or `EU`).	Yes
`credentials_json`	The complete contents of your Service Account key, formatted as a stringified JSON (escaped quotes and newlines).	Yes

💡

Some users may specify dataset_id in the format <project_id>.<dataset_id> (for example, my-test-project-84030.abc123), which is valid in BigQuery but not supported in Zenskar. Pass only the dataset name (abc123), as project_id is already handled separately.

Ensure that the Service Account used has the BigQuery User and BigQuery Data Editor roles.

Zenskar supports only JSON-format Service Account keys.

Notes

📘
Zenskar does not modify your existing BigQuery data or schema.

📘
Queries are executed using your service account’s permissions.

📘
For best performance, use partitioned or clustered tables when working with large datasets.