Atlan’s lineage generation is based on SQL-parsing. Parsing of Spark jobs is currently not supported. However, Databricks has released their Unity Catalog solution that enables lineage generation from Spark jobs that occur in Databricks. Atlan offers native integration with the Unity Catalog API to ingest lineage from Spark transformations in Databricks.
- What is column-level lineage?
- What do the numbers that show up in lineage mean?
- Can my team be notified if there is a change in dashboards downstream / notified if there is schema drift?
- Can Atlan send me an alert if a column drops?
- How does Atlan handle lineage from Spark jobs?
- Why is my Databricks lineage API not working?
- How to obtain upstream lineage if I connect to a Tableau data asset?
- How is the Atlan lineage graph depicted using Power BI measures?
- How can Atlan generate lineage upstream from the data warehouse layer?
- How does column lineage work?