Atlan’s lineage generation is based on SQL-parsing. Parsing of Spark jobs is not currently supported, however Databricks has released their Unity Catalog solution that enables lineage generation from Spark jobs that occur in Databricks — Atlan has a native integration with the Unity Catalog API to ingest lineage from Spark transformations in Databricks.
- Can my team be notified if there is a change in dashboards downstream / notified if there is schema drift?
- Can Atlan send me an alert if a column drops?
- How does Atlan handle lineage from spark jobs?
- My Databricks Lineage API doesn't seem to be working
- Could you obtain the asset's upstream lineage if you connect to a Tableau data asset?
- How is the Atlan lineage graph depicted using Power BI measures?
- How can Atlan generate lineage upstream from the data warehouse layer?
- How does Column Lineage work?
- How does say, a snowflake connection in Atlan know that the table A referenced in LookerML/sql in Looker is actually the same table A?
- What is a process?