How to crawl DataStax Enterprise

Once you have configured DataStax Enterprise, you can establish a connection between Atlan and DataStax Enterprise.

To crawl metadata from DataStax Enterprise, review the order of operations and then complete the following steps.

Select the source

To select DataStax Enterprise as your source:

1. In the top right of any screen, navigate to New and then click New Workflow.
2. From the list of packages, select DataStax Enterprise Assets and click the Setup Workflow button.

Provide credentials

• In Direct extraction, Atlan connects to DataStax Enterprise and crawls metadata directly.
• In Agent extraction, Atlan’s secure agent executes metadata extraction within the organization's environment.

Direct extraction method

To enter your DataStax Enterprise credentials:

1. For Host Name, enter the host name of your DataStax Enterprise instance.
2. For Port, enter the port number of your DataStax Enterprise instance.
3. For Authentication, Basic authentication, enter the Username and Password you use to log in to DataStax Enterprise.
4. (Optional) For SSL, keep the default Enabled to use HTTPS or click Disabled to use HTTP.
5. (Optional) For SSL certificate, this is only required if your DataStax Enterprise instance uses a self-signed or an internal CA SSL certificate, paste a supported SSL certificate in the recommended format.
6. At the bottom of the form, click the Test Authentication button to confirm connectivity to DataStax Enterprise using these details.
7. When successful, at the bottom of the screen click the Next button.

Agent extraction method

Atlan supports using a Secure Agent for fetching metadata from DataStax Enterprise. To use a Secure Agent, follow these steps:

1. Select the Agent tab.
2. Configure the DataStax Enterprise data source by adding the secret keys for your secret store. For details on the required fields, refer to the Direct extraction section.
3. Complete the Secure Agent configuration by following the instructions in the How to configure Secure Agent for workflow execution guide.
4. Click Next after completing the configuration.

Configure the connection

To complete the DataStax Enterprise connection configuration:

1. Provide a Connection Name that represents your source environment. For example, you might want to use values like production, development, gold, or analytics.
2. (Optional) To change the users able to manage this connection, change the users or groups listed under Connection Admins.
   🚨 Careful! If you do not specify any user or group, nobody will be able to manage the connection — not even admins.

Configure the crawler

Before running the DataStax Enterprise crawler, you can further configure it.

1. On the Metadata page, you can override the defaults for any of these options:
   • To select the DataStax Enterprise keyspaces you want to include in crawling, click Include Keyspaces. (This will default to all assets, if none are specified.)
   • To select the DataStax Enterprise keyspaces you want to exclude from crawling, click Exclude Keyspaces. (This will default to no assets, if none are specified.)
   • To have the crawler ignore DataStax Enterprise keyspaces based on a naming convention, specify a regular expression in the Exclude Keyspaces Regex field.
2. To check for any permissions or other configuration issues before running the crawler, click Preflight checks.

Once the crawler has completed running, you will see the assets in Atlan's asset page! 🎉