Lineage
Atlan uses Fivetran's log events from the Fivetran Platform Connector to generate lineage associated with Fivetran connectors. This is particularly useful for creating lineage between different tools, such as Salesforce and Snowflake.
Specifically, Atlan will:
- Create lineage between each data asset in Atlan that is associated with a Fivetran connector. (For example, between Salesforce objects and Snowflake tables.)
- Atlan creates Fivetran
Processobjects for each data asset that is replicated. - Atlan creates column-level lineage to connect the sources (inputs) and destinations (outputs) of each
Process. (For example, between Salesforce fields and Snowflake columns.)
- Atlan creates Fivetran
- Link each
Processto its corresponding connector in the Fivetran UI. - For any Fivetran sources or destinations that are not natively supported or are supported but have not been crawled yet, Atlan will create partial assets to provide you with a complete view of lineage.
Supported sources and destinations for Fivetran Platform Connector
Sources
Atlan's Fivetran Platform Connector integration supports all sources listed in Fivetran documentation for lineage.
For any sources that are not natively supported or are supported but have not been crawled yet, Atlan will create partial assets to provide you with a complete view of lineage. Note that Atlan will only catalog partial column assets and generate column-level lineage for SQL sources natively supported in Atlan.
Destinations
Atlan's Fivetran Platform Connector integration supports all destinations listed in Fivetran documentation for lineage.
For any destinations that are not natively supported or are supported but have not been crawled yet, Atlan will create partial assets to provide you with a complete view of lineage. Note that Atlan will only catalog partial column assets and generate column-level lineage for SQL sources natively supported in Atlan.
Note that for crawler configuration, only the destinations listed here are supported.
Connectors
Atlan maps the following metadata from Fivetran to its FivetranConnector asset type. This is only applicable to metadata crawled using the Fivetran Platform Connector.
| Source property | Atlan property | Where in Atlan |
|---|---|---|
connector.NAME |
fivetranConnectorConnectorName |
overview sidebar |
connector_type.ID |
fivetranConnectorConnectorType |
overview sidebar |
| dynamically created by script | fivetranConnectorConnectorURL |
overview sidebar |
destination.NAME |
fivetranConnectorDestinationName |
overview sidebar |
destination.TYPE |
fivetranConnectorDestinationType |
overview sidebar |
| dynamically created by script | fivetranConnectorDestinationURL |
overview sidebar |
connector.SIGNED_UP |
fivetranConnectorSyncSetupOn |
overview sidebar |
connector.SYNC_FREQUENCY |
fivetranConnectorSyncFrequency |
overview sidebar |
connector.PAUSED |
fivetranConnectorSyncPaused |
API only |
user.GIVEN_NAME + user.FAMILY_NAME
|
fivetranConnectorSyncSetupUserFullName |
API only |
connector.USER_EMAIL |
fivetranConnectorSyncSetupUserEmail |
API only |
incremental_mar. INCREMENTAL_ROWS |
fivetranConnectorMonthlyActiveRowsFree |
overview sidebar |
incremental_mar. INCREMENTAL_ROWS |
fivetranConnectorMonthlyActiveRowsPaid |
overview sidebar |
fivetranConnectorMonthlyActiveRowsFree + fivetranConnectorMonthlyActiveRowsPaid
|
fivetranConnectorMonthlyActiveRowsTotal |
API only |
| calculated by script | fivetranConnectorMonthlyActiveRowsChangePercentageFree |
API only |
| calculated by script | fivetranConnectorMonthlyActiveRowsChangePercentagePaid |
API only |
| calculated by script | fivetranConnectorMonthlyActiveRowsChangePercentageTotal |
overview sidebar |
| calculated by script | fivetranConnectorMonthlyActiveRowsFreePercentageOfAccount |
API only |
| calculated by script | fivetranConnectorMonthlyActiveRowsPaidPercentageOfAccount |
overview sidebar |
| calculated by script | fivetranConnectorMonthlyActiveRowsTotalPercentageOfAccount |
API only |
| dynamically generated by script | sourceURL |
overview sidebar |
connector_type.CREATED_AT |
sourceCreatedAt |
asset preview, profile, and filter, overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncId |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncStartedAt |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncFinishedAt |
overview sidebar |
calculated from LOG table |
fivetranLastSyncStatusfivetranConnectorLastSyncReason |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncTaskType |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncRescheduledAt |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncTablesSynced |
overview sidebar |
calculated from LOG table |
fivetranLastSyncRecordsUpdated |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncExtractTimeSeconds |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncExtractVolumeMegabytes |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncLoadTimeSeconds |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncLoadVolumeMegabytes |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncProcessTimeSeconds |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncProcessVolumeMegabytes |
overview sidebar |
calculated from LOG table |
fivetranConnectorLastSyncTotalTimeSeconds |
overview sidebar |