What does Atlan crawl from Airflow/OpenLineage?

Once you have integrated Airflow/OpenLineage, you can use connector-specific filters for quick asset discovery. The following filters are currently supported:

  • Status filter — last run status for an asset
  • Duration filter — last run duration for an asset

Atlan maps the following assets and properties from Airflow/OpenLineage.

DAGs

Atlan maps DAGs (directed acyclic graphs) from Airflow/OpenLineage to its AirflowDAG asset type.

Source property Atlan property
schedule_interval airflowDagSchedule
timetable airflowDagScheduleDelta
tags airflowTags
version airflowRunVersion
openlineageAdapterVersion airflowRunOpenLineageVersion
runid airflowRunName
run_type airflowRunType
eventTime airflowRunStartTime
eventTime airflowRunEndTime
eventType airflowRunOpenLineageState

Tasks

Atlan maps tasks from Airflow/OpenLineage to its AirflowTask asset type.

Source property Atlan property
retries airflowTaskRetryNumber
pool airflowTaskPool
pool_slots airflowTaskPoolSlots
queue airflowTaskQueue
priority_weight airflowTaskPriorityWeight
trigger_rule airflowTaskTriggerRule
operator_class airflowTaskOperatorClass
dag_id airflowDagName
conn_id airflowTaskConnectionId
sql airflowTaskSql
tags airflowTags
version airflowRunVersion
openlineageAdapterVersion airflowRunOpenLineageVersion
runid airflowRunName
run_type airflowRunType
eventTime airflowRunStartTime
eventTime airflowRunEndTime
eventType airflowRunOpenLineageState

Related articles

Was this article helpful?
0 out of 0 found this helpful