The following connectors have built-in lineage generation:
- Inside a data warehouse (table-to-table, view-to-table — this goes down to a column level)
- Data warehouse — Databricks, Snowflake, Amazon Redshift, and Google BigQuery
- The miner packages in the Atlan marketplace bring in lineage
- Snowflake, Google BigQuery, and Amazon Redshift lineage are built by parsing queries in query history
- Unity Catalog APIs provide Databricks lineage (only if the Databricks workspace is Unity Catalog-enabled — which is still in preview from Databricks)
- Lineage from data warehouse to BI tool
- BI tool — Power BI, Tableau, and Looker
- Data warehouse — all the aforementioned sources
- dbt — we're able to extract table-level lineage from dbt:
- Column-level lineage for dbt models is only generated from model to model and model to metric
- Since sources do not have columns to map to, column-level lineage is not available for source to model
- Data warehouse and data lake sources are covered by dbt lineage, and we can use dbt to extract AWS Glue's lineage
If you see a connector you'd like to manually build lineage for that is not automated, you can do it by utilizing our APIs.
