How to manage Databricks tags

Atlan allows you to import your Databricks tags, update your Databricks assets with the imported tags, and push the tag updates back to Databricks:

• Import tags — crawl Databricks tags from Databricks to Atlan
• Reverse sync — sync Databricks tag updates from Atlan to Databricks

Once you've imported your Databricks tags to Atlan:

• Your Databricks assets in Atlan will be automatically enriched with their Databricks tags.
• Imported Databricks tags will be mapped to corresponding Atlan tags through case-insensitive name match — multiple Databricks tags can be matched to a single tag in Atlan.
• You can also attach Databricks tags, including tag values, to your Databricks assets in Atlan — allowing you to categorize your assets at a more granular level.
• You can filter your assets by Databricks tags and tag values.
• You can enable reverse sync to push any tag updates for your Databricks assets back to Databricks — including tag values added to assets in Atlan.

Prerequisites

You must have a Unity Catalog-enabled workspace and SQL warehouse configured to import Databricks tags in Atlan.

Before you can import tags from and push tag updates to Databricks using personal access token, AWS service principal, or Azure service principal authentication, you will need to do the following:

• Ensure that you have a Unity Catalog-enabled workspace and a SQL warehouse configured.
• Create tags or have existing tags in Databricks.
• Grant permissions to import tags from and push tag updates to Databricks.

Import Databricks tags to Atlan

You can import your Databricks tags to Atlan through one-way tag sync. The synced Databricks tags will be matched to corresponding tags in Atlan through case-insensitive name match and your Databricks assets will be enriched with their synced tags from Databricks.

To import Databricks tags to Atlan, you can either:

• Create a new Databricks workflow and configure the crawler to import tags.
• Modify the crawler's configuration for an existing Databricks workflow to change Import Tags to Yes.

Once the crawler has completed running, tags imported from Databricks will be available to use for tagging assets! 🎉

View Databricks tags in Atlan

Once you've imported your Databricks tags, you will be able to view and manage your Databricks tags in Atlan.

To view Databricks tags:

1. From the left menu of any screen, click Governance.
2. Under the Governance heading of the Governance center, click Tags.
3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
4. From the left menu under Tags, select a synced tag.
5. In the Overview section, you can view a total count of synced Databricks tags. To the right of Overview, click Synced tags to view additional details — including tag name, description, tag values, total count of linked assets, connection, database, and schema names, and timestamp for last synced.
6. (Optional) Click the Linked assets tab to view linked assets for your Databricks tag.
7. (Optional) In the top right, click the pencil icon to add a description and change the tag icon. You cannot rename tags synced from Databricks.

Push tag updates to Databricks

You can enable reverse sync for your imported Databricks tags in Atlan and push all tag updates for your Databricks assets back to source. Once you have enabled reverse sync, any Databricks assets with tags updated in Atlan will also be updated in Databricks.

To enable reverse sync for imported Databricks tags:

1. From the left menu of any screen, click Governance.
2. Under the Governance heading of the Governance center, click Tags.
3. (Optional) Under Tags, click the funnel icon to filter tags by source type. Click Databricks to filter for tags imported from Databricks.
4. In the left menu under Tags, select a synced Databricks tag — synced tags will display the Databricks icon next to the tag name. 
5. On your selected tag page, to the right of Overview, click Synced tags.
6. Under Synced tags, in the upper right, turn on Enable reverse sync to synchronize tag updates from Atlan to Databricks.
7. In the corresponding confirmation dialog, click Yes, enable it to enable reverse tag sync or click Cancel.

Now when you attach Databricks tags to your Databricks assets in Atlan, these tag updates will also be pushed to Databricks! 🎉