How to set up playbooks

A common question that data teams often face is how to automate metadata at scale.

Having started out as a data team ourselves, we know that automating repetitive tasks can help data teams maximize the value they provide to their organization. One way of doing so is through Atlan's playbooks!

Playbooks help power metadata automation for your data assets in Atlan. Users can create rule-based automations at scale and update metadata in bulk, helping streamline their workflows.

You can update the following asset metadata using playbooks:

• Certificates
• Descriptions
• Owners
• Terms
• Tags
• Custom metadata

For example, imagine your organization needs to transfer ownership of several data assets. Instead of your data team manually updating the ownership of each and every asset, you could create a playbook to automate this process and update the metadata of your assets at scale.

To create a playbook, complete the following steps.

Create a playbook

To create a playbook in Atlan:

1. In the left menu in Atlan, click Governance.
2. Under the Governance heading of the Governance center, click Playbooks.
3. Click Create New to get started.
4. In the Create new playbook dialog box, enter the following details:
   1. For Name, enter a name for the task to be accomplished — for example, Update ownership. (Atlan recommends that the length of a playbook name must be no longer than 46 characters.)
   2. (Optional) For Description, enter a description.
   3. (Optional) Select an icon for your playbook.
5. Click Create to save your playbook.

Set up rules as filters

To set up rules as filters for your playbook:

1. In the Build Rules page of your playbook, click Filters.
2. For name, add a name to your filter.
3. To set a matching condition for the filters, select Match all or Match any. Match all will logically AND the criteria, while Match any will logically OR the criteria.
4. For Attributes, select a relevant option:
   • For this example, we'll click Connection and then select a Snowflake connection. (Optional) To further refine your asset selection:
     1. Click All databases to filter by databases in a selected connection.
     2. Click All schemas to filter by schemas in a selected connection.
   • Click Connector to filter assets by supported connectors.
   • Click Asset type to filter by specific asset types — for example, tables, columns, queries, glossaries, and more.
   • Click Certificate to filter assets by certification status.
   • Click Owners to filter assets by asset owners.
   • Click Tags to filter assets by your tags in Atlan, including imported Snowflake and dbt tags. 
   • Click Glossary, terms, & categories to filter by a specific glossary or category to bulk update all the nested terms or by multiple glossaries and categories.
   • Click Linked terms to filter assets by linked terms.
   • Click Schema qualified Name to filter assets by the qualified name of a given schema.
   • Click Database qualified Name to filter assets by the qualified name of a given database.
   • Click dbt to filter assets by dbt-specific filters and then select a dbt Cloud or dbt Core filter.
   • Click Properties to filter assets by common asset properties.
   • Click Usage to filter assets by usage metrics.
   • Click Monte Carlo to filter assets by Monte Carlo-specific filters.
   • Click Soda to filter assets by Soda-specific filters.
   • Click Table/View to filter tables or views by row count, column count, or size.
   • Click Column to filter columns by column-specific filters, including parent asset type or name, data type, or column keys.
   • Click Process to filter lineage processes by the SQL query.
   • Click Query to filter assets by associated visual queries.
   • Click Measure to filter Microsoft Power BI measures using the external measures filter.
5. For Operator, select Is one of for values to include or Is not for values to exclude. Depending on the selected attribute(s), you can also choose from additional operators:
   • Select Equals (=) or Not Equals (!=) to include or exclude assets through exact match search.
   • Select Starts With or Ends With to filter assets using the starting or ending sequence of values.
   • Select Contains or Does not contain to find assets with or without specified values contained within the attribute.
   • Select Pattern to filter assets using supported Elastic DSL regular expressions.
   • Select Is empty to filter assets with null values.
6. For Values, select the relevant values. The values will vary depending on the selected attributes.
7. (Optional) To add more filters, click Add filter and select Filter to add individual filters or Filter Group to nest more filters in a group.
8. (Optional) To view all the assets that match your rules, in the Filters card, click View all for a preview.

Select the actions

To select the actions to be performed based on your rules:

1. In the Build Rules page of your playbook, click Actions.
2. For Select Action, select the relevant metadata option to update:
   • Click Certificate to update the certification status of assets to Verified, Draft, Deprecated, or No certificate.
   • Click Description to update the description of your assets.
   • Click Owners to add, remove, or replace asset owners. In this example, we'll update the ownership of the assets.
   • Click Terms to add terms to your assets or remove or replace them from linked assets.
   • Click Tags to add tags to your assets or remove or replace them from tagged or propagated assets.
   • Click any custom metadata structure and then select a custom metadata property to update or unlink it from your assets.
3. For Select operator, select the relevant option. The operators will vary depending on the selected action.
4. For Values, select the relevant option(s). The values will vary depending on the selected actions.
5. (Optional) To add more actions, click Add Action.

Run the playbook

If you'd like to continue working on your playbook, you can save it as a draft. If your playbook is ready, you can proceed to running it.

To run the playbook:

1. You can either:
   • To run the playbook once immediately, click Run once.
   • To schedule the playbook to run hourly, daily, weekly, or monthly, click Schedule and choose the preferred frequency, timezone, and time.
     🚨 Careful! If you're scheduling multiple playbooks, Atlan recommends spacing out the schedules as much as possible to minimize any overlap between the playbook workflow runs.
2. Click Complete to run the playbook.
3. In the resulting screen, click Go to profile to view your playbook profile.

Once your playbook has completed its run, you will see the metadata updated for your assets! 🎉