How to set up Aiven Kafka

Atlan supports the S3 extraction method for fetching metadata from Aiven Kafka. This method uses Atlan's kafka-extractor tool to fetch metadata.

Create user in Aiven Kafka

To create a new user for extracting metadata from Aiven Kafka:

1. Log in to your Aiven console and select your active cluster.
2. From the upper right of the cluster Overview page, click the Users tab to create a new user:
   1. For Create a service user, under Username, enter a name for the new user and then click Add service user.
   2. The new user will be listed under Service users, on the Users page. Copy the username and password for the new user and store them in a secure location.
   3. (Optional) If using client certificate authentication, copy the access key and access certificate and store them in a secure location.
3. From the upper right of the cluster Overview page, click the Access Control List (ACL) tab to add a new ACL grant:
   1. Under Access Control List (ACL), for ACL Type, click ACL For Topic.
   2. For Add access control entry, enter the following details:
      1. For Username, enter the username you created for the new user.
      2. For Topic, enter an asterisk * to include all topics.
      3. From the Permission dropdown, select Admin — learn more about ACL permission mapping.
      4. Click Add entry to save your selections.
4. Navigate to the Overview tab, copy or download the CA Certificate and store the details in a secure location.