Atlan has 2 API endpoints to generate lineage:
Atlan generates upstream lineage in the following ways:
- Custom jobs:
- Constraint — these jobs do not have an interface for Atlan to extract lineage. For example, custom Python jobs that move data from RDBMS to S3 to Snowflake.
- Recommended path — use the lineage API to push source-to-target mappings to Atlan.
- Data integration tools:
- Constraint — these tools have historically not had metadata APIs for Atlan to extract lineage.
- Recommended path — Atlan has released support for data integration tools that have released metadata APIs, such as Fivetran.
- Orchestration tools:
- Constraint — you need to run DAGs or jobs to catalog assets and lineage in Atlan, unlike other workflows that catalog assets after a workflow run.
- Recommended path — use the lineage API to either create lineage directly or using OpenLineage.