How to crawl Fivetran

Once you have configured the Fivetran permissions, you can establish a connection between Atlan and Fivetran.

To enrich Atlan with metadata from Fivetran, review the order of operations and then complete the following steps.

Select the source

To select Fivetran as your source:

1. In the top right of any screen, navigate to New and then click New Workflow.
2. From the list of packages, select Fivetran Enrichment and then click Setup Workflow.

Provide credentials

In order to run this package, you must ensure the following:

• The Fivetran Platform Connector is set up and has run successfully in Fivetran at least once.
• Fivetran logs are stored in a destination supported by Atlan.
• The above destination has been crawled in Atlan.

To use the Fivetran Platform Connector:

1. For Atlan Connection, Atlan will use the credentials of your selected connection to read the Fivetran Platform Connector tables associated with that connection. You can either:
   • Create a connection in Atlan for the destination warehouse you configured while setting up the Fivetran Platform Connector in Fivetran. This connection in Atlan must have access to the Fivetran tables created by the Fivetran Platform Connector. Atlan supports the following destinations:
     • Amazon Redshift
     • Databricks
     • Google BigQuery
     • PostgreSQL
     • Snowflake
   • If you have already created a connection in Atlan, select the connection to extract. (To select a connection, the crawler must have already run for a supported destination.)
     🚨 Careful! If you have an existing connection, you must ensure that the user or other access permissions configured for that connection allow access to the Fivetran tables created by the Fivetran Platform Connector or update them accordingly.
2. For Fivetran Platform Schema, select the destination schema where Fivetran logs are stored. You can only select one schema and must ensure that the connection above has access to all Fivetran log tables stored in this destination schema.
3. Once successful, at the bottom of the screen, click Next.

Configure the connection

To complete the Fivetran connection configuration:

1. Provide a Connection Name that represents your source environment. For example, you might use values like production, development, gold, or analytics.
2. (Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins.
   🚨 Careful! If you do not specify any user or group, nobody will be able to manage the connection — not even admins.

Run the enrichment

You can now enrich Atlan with Fivetran metadata:

1. To check for any permissions or other configuration issues before running the enrichment, click Preflight checks.
2. You can either:
   • To run the crawler once immediately, at the bottom of the screen, click the Run button.
   • To schedule the crawler to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule & Run button.

Once the enrichment has completed running, you will see lineage extended upstream from your data platform, warehouse, or lake! 🎉