Azure Private Link creates a secure, private connection between services running in Azure. This document describes the steps to set this up between Databricks and Atlan.
🤓 Who can do this? You will need Databricks support, and probably your Databricks administrator involved — you may not have access or the tools to run these tasks.
Prerequisites
- Your Databricks instance must be Azure-managed and created from the Azure marketplace.
For all details, see Databricks documentation.
Notify Atlan support
Provide Atlan support with the following information:
- Resource ID of your Azure-managed Databricks instance — the resource ID will be in this format:
/subscriptions/<subscriptionID>/resourceGroups/azure-databricks/providers/Microsoft.Databricks/workspaces/<databricks-workspace>.
There are additional steps that Atlan will need to complete. Once the Atlan team has confirmed that the configuration is ready, please continue with the remaining steps.
Approve the endpoint connection request
To approve the endpoint connection request from Atlan:
- Open your Azure-managed Databricks workspace.
- In the left menu, click Networking and then click the Private endpoint connections tab.
- From the list of endpoints, search for the endpoint connection request from Atlan. In the Connection state column for the Atlan endpoint connection, click the Approve button to approve the request .
Once the endpoint connection from Atlan has been approved, the status of the private endpoint in Atlan will change to Approved.
When you use this endpoint in the configuration for crawling Databricks, Atlan will connect to Databricks over Azure Private Link.