Why is the metadata getting lost when migrating from Snowflake to dbt?

Have more questions? Submit a request

As we migrate our Snowflake views to dbt, it seems we're losing the metadata associated with those views. For example, after the migration to dbt, a verified view became unverified and lost all the related terms and the README. Is this preventable?

Migration from Snowflake to dbt turns these from views into tables, which changes the nature of the asset in Atlan. As such, the old views are archived (soft-deleted).

Atlan will soft-delete/archive an asset in the following scenarios:

  1. If the asset type changes from a view to table or vice versa
  2. If the asset location (hierarchy) changes, i.e. the asset moves to a different schema or a database
  3. If the asset is dropped / deleted in the warehouse

In the first two cases, Atlan archives the original asset and creates a new one in its place. This happens because properties like the asset type and the qualifiedName which uniquely define the asset undergo a change. Atlan would then consider the newly crawled asset as a new one, with its own unique id.

As for the Readme: while it's been delinked from the archived version of the asset, the contents can be retrieved using our APIs

While no glossary terms will be attached to the archived asset, you may see suggestions on the new asset based the former's previous terms.

Related articles

Was this article helpful?
0 out of 0 found this helpful