- Inside a data warehouse (table-to-table, view-to-table — this goes down to a column level)
-
- Data warehouse = Databricks, Snowflake, Redshift, and BigQuery
- The miner packages that you can find in the Atlan Workflow marketplace are what bring this in:
- Snowflake, BigQuery, and Redshift lineage are built by parsing queries in query history.
- Unity catalog APIs provide Databricks lineage (only if the Databricks workspace is Unity catalog enabled — which is still in preview from Databricks)
- Lineage from Data warehouse to BI tool
-
- BI tool = Power BI, Tableau, Looker
- Data warehouse = all the aforementioned sources
- dbt -- we're able to extract table level lineage from dbt
- Data warehouse and data lake sources are covered by dbt lineage, and we can use dbt to extract AWS Glue's lineage.
If you see a connector you'd like to manually build lineage for that is not automated, you can do it by utilizing our API:
