How to monitor connectivity

Have more questions? Submit a request

Atlan runs its crawlers through an orchestrated set of automated tasks.

To monitor these orchestrated set of tasks follow these steps.

Monitor the crawling process

You can visualize the individual tasks a workflow runs as a directed acyclic graph (or "DAG").

To visualize the crawling process:

  • When running a workflow immediately, you will be redirected to the monitoring page within 5 seconds.
  • At any other moment:
    1. From the left menu, navigate to the Workflow Center:
      • By default, workflow runs from the last 24 hours will be shown.
      • (Optional) Use the filters along the top to narrow down to the workflow run you want to monitor.
    2. From the Workflow run history table, click the workflow run you want to check.

On the left of the screen under the Summary tab you can also see:

  • The current status of the workflow run.
  • The start and finish time of the workflow run.
  • The elapsed time (duration) of the workflow run.
  • Who triggered the workflow run and how (manually or automatically).

Identify errors

If a crawler fails due to an error, Atlan will show where the failure occurred in the visualization.

To review the failure of any workflow with an error:

  1. Open the workflow run visualization (using either option above).
  2. Under the Summary tab on the left of the screen, click the View Failed tasks button.

Atlan will take you to the Failed Tasks tab on the left of the screen. Here you can review details about the specific activity or activities that failed.

Review log files

Each task in the DAG may produce a log file containing additional details.

To review the log file for a specific activity:

  1. Click the task (activity node) in the DAG visualization.
  2. Open the Failed Tasks tab of a workflow run visualization (see steps above).
  3. To the right of each failed step, click the Logs button.

If there are any logs available, Atlan will display them on the screen.

πŸ’ͺ Did you know? Not every failed activity will produce a log. Look at the Message field of failed tasks for ideas about what went wrong when there is no log file available.

Related articles

Was this article helpful?
1 out of 1 found this helpful