How to set up Microsoft Azure Data Factory

Atlan supports service principal authentication for fetching metadata from Microsoft Azure Data Factory. This method requires a client ID, client secret, and tenant ID to fetch metadata.

Register app with Microsoft Entra ID

You will need to register your service principal application with Microsoft Entra ID and note down the values of the tenant ID, client ID, and client secret.

To register your app with Microsoft Entra ID:

1. Log in to the Azure portal.
2. In the search bar, search for Microsoft Entra ID, and select it from the dropdown list.
3. From the left menu of the Microsoft Entra ID page, click App registrations.
4. From the toolbar on the App registrations page, click + New registration.
5. On the Register an application page, for Name, enter a name for your service principal application and then click Register.
6. On the homepage of your newly created application, from the Overview screen, copy the values for the following fields and store them in a secure location:
   • Application (client) ID
   • Directory (tenant) ID
7. From the left menu of your newly created application page, click Certificates & secrets.
8. On the Certificates & secrets page, under Client secrets, click + New client secret.
9. In the Add a client secret screen, enter the following details:
   1. For Description, enter a description for your client secret.
   2. For Expiry, select when the client secret will expire.
   3. Click Add.
10. On the Certificates & secrets page, under Client secrets, for the newly created client secret, click the clipboard icon to copy the Value and store it in a secure location.

Set permissions

You will need to add the service principal to the Reader role. This will allow the service principal read-only access to your Microsoft Azure Data Factory account.

To add the service principal to the Reader role:

1. Log in to the Azure portal.
2. Open the menu and search for or select Data factories.
3. On the Data factories page, select the data factory you want to crawl in Atlan.
4. From the left menu of your data factory page, click Access control (IAM).
5. From the tabs along the top of the Access control (IAM) page, click Add and then click Add role assignment.
6. On the Add role assignment page, configure the following:
   1. In the Roles tab, from the list of roles under Job function roles, select Reader — this allows read-only access to your data factory — and then click Next. You will need to assign this role to all the data factories you want to crawl in Atlan.
   2. In the Members tab, enter the following details:
      1. For Assign access to, click User, group, or service principal.
      2. For Members, click + Select members and then select the service principal you created. Click Next to proceed to the next step.
   3. In the Review + assign tab, click Review + assign to add role assignment.

Atlan will extract metadata from all the data factories you specified in your Microsoft Azure Data Factory account with Reader access.