What does Atlan crawl from Airflow/OpenLineage?

Have more questions? Submit a request
πŸ§ͺ Preview feature! This feature is available for your experimentation, and we'd love your feedback. It may change before its final generally-available form. If you'd like to participate in the preview, reach out to your customer success manager for more information.

Once you have integrated Airflow/OpenLineage, you can use connector-specific filters for quick asset discovery. The following filters are currently supported:

  • Status filter β€” last run status for an asset
  • Duration filter β€” last run duration for an asset

Atlan maps the following assets and properties from Airflow/OpenLineage.

DAGs

Atlan maps DAGs (directed acyclic graphs) from Airflow/OpenLineage to its AirflowDAG asset type.

Source property Atlan property
schedule_interval airflowDagSchedule
timetable airflowDagScheduleDelta
tags airflowTags
version airflowRunVersion
openlineageAdapterVersion airflowRunOpenLineageVersion
runid airflowRunName
run_type airflowRunType
eventTime airflowRunStartTime
eventTime airflowRunEndTime
eventType airflowRunOpenLineageState

Tasks

Atlan maps tasks from Airflow/OpenLineage to its AirflowTask asset type.

Source property Atlan property
retries airflowTaskRetryNumber
pool airflowTaskPool
pool_slots; airflowTaskPoolSlots
queue airflowTaskQueue
priority_weight airflowTaskPriorityWeight
trigger_rule airflowTaskTriggerRule
operator_class airflowTaskOperatorClass
dag_id airflowDagName
conn_id airflowTaskConnectionId
sql airflowTaskSql
tags airflowTags
version airflowRunVersion
openlineageAdapterVersion airflowRunOpenLineageVersion
runid airflowRunName
run_type airflowRunType
eventTime airflowRunStartTime
eventTime airflowRunEndTime
eventType airflowRunOpenLineageState

Related articles

Was this article helpful?
0 out of 0 found this helpful