Once you have integrated Google Cloud Composer/OpenLineage, you can use connector-specific filters for quick asset discovery. The following filters are currently supported:
- Status filter — last run status for an asset
- Duration filter — last run duration for an asset
Atlan maps the following assets and properties from Google Cloud Composer/OpenLineage. Asset lineage support depends on the list of operators supported by OpenLineage.
DAGs
Atlan maps DAGs (directed acyclic graphs) from Google Cloud Composer/OpenLineage to its AirflowDAG
asset type.
Source property | Atlan property |
---|---|
schedule_interval |
airflowDagSchedule |
timetable |
airflowDagScheduleDelta |
tags |
airflowTags |
version |
airflowRunVersion |
openlineageAdapterVersion |
airflowRunOpenLineageVersion |
runid |
airflowRunName |
run_type |
airflowRunType |
eventTime |
airflowRunStartTime |
eventTime |
airflowRunEndTime |
eventType |
airflowRunOpenLineageState |
query |
airflowTaskSql |
group_id |
airflowTaskGroupName |
Tasks
Atlan maps tasks from Google Cloud Composer/OpenLineage to its AirflowTask
asset type.
Source property | Atlan property |
---|---|
retries |
airflowTaskRetryNumber |
pool |
airflowTaskPool |
pool_slots |
airflowTaskPoolSlots |
queue |
airflowTaskQueue |
priority_weight |
airflowTaskPriorityWeight |
trigger_rule |
airflowTaskTriggerRule |
operator_class |
airflowTaskOperatorClass |
dag_id |
airflowDagName |
conn_id |
|
sql |
airflowTaskSql |
tags |
airflowTags |
version |
airflowRunVersion |
openlineageAdapterVersion |
airflowRunOpenLineageVersion |
runid |
airflowRunName |
run_type |
airflowRunType |
eventTime |
airflowRunStartTime |
eventTime |
airflowRunEndTime |
eventType |
airflowRunOpenLineageState |