How can Atlan generate upstream lineage from the data warehouse layer?

Atlan has 2 API endpoints to generate lineage:

• lineage between assets
• column-level lineage

Atlan generates upstream lineage in the following ways:

• Custom jobs:
  • Constraint — these jobs do not have an interface for Atlan to extract lineage. For example, custom Python jobs that move data from RDBMS to S3 to Snowflake.
  • Recommended path — use the lineage API to push source-to-target mappings to Atlan.
• Data integration tools: 
  • Constraint — these tools have historically not had metadata APIs for Atlan to extract lineage.
  • Recommended path — Atlan has released support for data integration tools that have released metadata APIs, such as Fivetran.
• Orchestration tools: 
  • Constraint — you need to run DAGs or jobs to catalog assets and lineage in Atlan, unlike other workflows that catalog assets after a workflow run.
  • Recommended path — use the lineage API to either create lineage directly or using OpenLineage.