If an asset is deleted via API, will workflows recreate the asset on the next

Have more questions? Submit a request

If an asset is deleted via API using the either delete asset by guid or the bulk endpoint, will workflows recreate the asset on next run? What is the expected behavior?

Atlan works on the principle of reflecting exactly what's in the source. Any deletes being executed at the source will results in assets within Atlan being archived (i.e. soft deleted) as well.

Since our workflows are meant to execute in this sync, our recommendation for data sources that have connectors is generally that updates/deletes be managed entirely by the source workflows.

Considering our workflows also run differential crawls, i.e. any delta change on assets is what's crawled and updated in Atlan (this helps deliver faster runtimes and performant workflows), the following it what will happen if any explicit delete is done using APIs:

Archiving (Soft delete):

  • The asset moves to an Archived state
  • The next workflow run will not unarchive/restore the asset automatically if the source asset does not undergo a change to one of the source metadata properties
  • The next workflow run will unarchive/restore the asset in its complete form, if one of the source properties changes (ex. a new column was added, etc.)

Note: Using the restore endpoint on the asset will unarchive the asset as well.

Hard delete:

  • The asset is completely removed from the metastore
  • The next workflow run will not unarchive/restore the asset automatically if the source asset does not undergo a change to one of the source metadata properties
  • The next workflow run will unarchive/restore the asset in a partial form, if one of the source properties changes (ex. a new column was added, etc.)
    • What does partial form mean? The asset will come back with only that one property/key present (ex. if a new column is added, the table will only have that one column - since all other columns were hard deleted). This will lead to the asset being incomplete and showing an incorrect representation of the source.

Note: Using the restore endpoint will not unarchive the asset.

Hence, our recommendation for this to be managed by workflows. We provisioned the delete and restore endpoints for assets created via APIs (i.e. not being crawled via a workflow).

 

 

 

Related articles

Was this article helpful?
0 out of 0 found this helpful