Integrating Databricks

Capabilities

Cyera's Databricks integration provides the following capabilities:

Product	Capability
Cyera Privacy	Request Types Access, Deletion, Do Not Sell/Share, Identifiers Identifier Categories Any
Live Data Map	Data Discovery

Before You Start

To successfully configure this integration, please ensure you have sufficient privileges:

• Databricks User Role: Admin
• Secrets Manager: Write Access

Connecting with RM Agent

The Cyera Privacy Agent allows you to automate Data Subject Requests by connecting to internal systems within your network, and without requiring ingress from the public network.

The Agent connects to your Databricks instance using least privileged credentials that you will create and store in a secrets manager. When configuring the Databricks integration in Cyera, only the location of that secrets manager entry will be referenced (e.g., AWS Secrets Manager ARN), which ensures that no secrets are shared directly with Cyera.

Prerequisites

Before you can connect to Databricks, ensure the following:

• RM Agent is deployed and connected in Cyera.
• Network is configured to allow the Agent to connect with the Databricks instance.

Create and Store Credentials

1. In Databricks, create a new user for the agent. Only grant the minimum necessary permissions for executing the request.

2. Configure the following JSON key-value pairs:

   {
     "host": "<Databricks instance host name>",
     "http_path": "<HTTP path to DBSQL endpoint (e.g. /sql/1.0/endpoints/...) or to a DBR interactive cluster (e.g. /sql/protocolv1/o/...)>",
     "access_token": "<HTTP Bearer access token, e.g. Databricks Personal Access Token>"
   }

3. Store the JSON value in your secrets manager with an entry name like cyera.rm-agent.databricks.

4. Ensure that the agent is configured to retrieve the value of this secrets manager entry.

Add the Agent Integration

1. In Cyera, navigate to Agents and select your Agent.
2. In the top right, select Add New Integration and search for Databricks.
3. Under Enabled Capabilities and Enabled Identifiers, select only those that will be used for this integration.
4. Enter the Credentials Location (e.g. AWS Secrets Manager ARN).
5. Select the Data Retrieval behavior for deletion requests.
   warning

   When using Retrieve Data, the data reviewed may not be exactly what is deleted due to the access and deletion logic executing separately!

6. Under Agent Query Configuration, add request logic to be executed within Databricks for all enabled request types.
   Query Parameter Format

   Use named paramstyle when formatting query parameters with identifiers (e.g., email, user_id).

   Example:

   SELECT * FROM users WHERE email = :email

7. Finally, select Configure Integration. Wait a few moments to ensure that the connection is successful. For failed connections, review the Agent container logs for additional details.