You can take the following reasons into consideration when accounting for longer workflow runtimes:
- Extracting a high volume of assets from the source may impact how long the workflows take to complete.
- Atlan workflows run differential crawls, allowing them to only bring in differential or delta changes to assets at source. This ensures that they run more quickly. Additionally, parallelization has been included to ensure further runtime optimization.
- In some cases, the workflow runtime can exceed the usual duration:
- There may be more transformations on some days, hence more delta changes to be synced to Atlan.
- In the event that one of these conversions also involves a delete operation, Atlan will archive the removed assets. The runtime of the overall workflow can be impacted because this archival activity often takes longer to complete.
Additionally, you can review some general guidelines and best practices for running workflows, see How to order workflows.