How to manage Databricks tags

Premium feature! This feature will be a paid addition. Reach out to your customer success manager for more information. You must have a Unity Catalog-enabled workspace to import Databricks tags in Atlan.

Atlan allows you to import your Databricks tags, update your Databricks assets with the imported tags, and push the tag updates back to Databricks:

  • Import tags — crawl Databricks tags from Databricks to Atlan
  • Reverse sync — sync Databricks tag updates from Atlan to Databricks

Once you've imported your Databricks tags to Atlan:

  • Your Databricks assets in Atlan will be automatically enriched with their Databricks tags.
  • Imported Databricks tags will be mapped to corresponding Atlan tags through case-insensitive name match — multiple Databricks tags can be matched to a single tag in Atlan.
  • You can also attach Databricks tags, including tag values, to your Databricks assets in Atlan — allowing you to categorize your assets at a more granular level.
  • You can filter your assets by Databricks tags and tag values.
  • You can enable reverse sync to push any tag updates for your Databricks assets back to Databricks — including tag values added to assets in Atlan.
💪 Did you know? Enabling reverse sync will only update existing tags in Databricks. It will neither create nor delete any tags in Databricks.

Prerequisites

You must have a Unity Catalog-enabled workspace and SQL warehouse configured to import Databricks tags in Atlan. You can import tags using any of the following authentication methods:

Personal access token authentication

Additional privileges are only required when using the personal access authentication method for fetching metadata.

Before you can import tags from Databricks, you will need to do the following:

AWS service principal authentication

Before you can import tags from Databricks, you will need to do the following:

Azure service principal authentication

Before you can import tags from Databricks, you will need to do the following:

Import Databricks tags to Atlan

🤓 Who can do this? You will need to be an admin user in Atlan to import Databricks tags to Atlan. You will also need to work with your Databricks administrator to grant permissions to import tags from Databricks — you may not have access yourself.

You can import your Databricks tags to Atlan through one-way tag sync. The synced Databricks tags will be matched to corresponding tags in Atlan through case-insensitive name match and your Databricks assets will be enriched with their synced tags from Databricks.

To import Databricks tags to Atlan, you can either:

Once the crawler has completed running, tags imported from Databricks will be available to use for tagging assets! 🎉

View Databricks tags in Atlan

Once you've imported your Databricks tags, you will be able to view and manage your Databricks tags in Atlan.

 

To view Databricks tags:

  1. From the left menu of any screen, click Governance.
  2. Under the Governance heading of the Governance center, click Tags.
  3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
  4. From the left menu under Tags, select a synced tag.
  5. In the Overview section, you can view a total count of synced Databricks tags. To the right of Overview, click Synced tags to view additional details — including tag name, description, tag values, total count of linked assets, connection, database, and schema names, and timestamp for last synced.
  6. (Optional) Click the Linked assets tab to view linked assets for your Databricks tag.
  7. (Optional) In the top right, click the pencil icon to add a description and change the tag icon. You cannot rename tags synced from Databricks.

Push tag updates to Databricks

🤓 Who can do this? Any admin or member user in Atlan can configure reverse sync for tag updates to Databricks. You will also need to work with your Databricks administrator to grant additional permissions to push updates — you may not have access yourself.

You can enable reverse sync for your imported Databricks tags in Atlan and push all tag updates for your Databricks assets back to source. Once you have enabled reverse sync, any Databricks assets with tags updated in Atlan will also be updated in Databricks.

 

To enable reverse sync for imported Databricks tags:

  1. From the left menu of any screen, click Governance.
  2. Under the Governance heading of the Governance center, click Tags.
  3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
  4. In the left menu under Tags, select a synced Databricks tag — synced tags will display the Databricks icon next to the tag name. 
  5. On your selected tag page, to the right of Overview, click Synced tags.
  6. Under Synced tags, in the upper right, turn on Enable reverse sync to synchronize tag updates from Atlan to Databricks.
  7. In the corresponding confirmation dialog, click Yes, enable it to enable reverse tag sync or click Cancel.

Now when you attach Databricks tags to your Databricks assets in Atlan, these tag updates will also be pushed to Databricks! 🎉

💪 Did you know? Enabling reverse sync will not trigger any updates in Databricks until synced tags are attached to Databricks assets in Atlan.

Related articles

Was this article helpful?
0 out of 0 found this helpful