Once you have crawled assets from Teradata, you can mine its query history to construct lineage.
To mine lineage from Teradata, review the order of operations and then complete the following steps.
Select the miner
To select the Teradata miner:
- In the top right of any screen, navigate to +New and then click New workflow.
- Under Marketplace, from the filters along the top, click Miner.
- From the list of packages, select Teradata Miner and then click Setup Workflow.
Configure the miner
To configure the Teradata miner:
- For Connection, select the connection to mine. (To select a connection, the crawler must have already run.)
- For Miner Extraction Method, choose your extraction method:
- In Query History, Atlan connects to your database and mines query history directly.
- In Offline, you will need to first mine query history yourself and make it available in S3. This method uses Atlan's teradata-miner tool to mine query history.
- For Start date, choose the earliest date from which to mine query history.
💪 Did you know? The miner restricts you to only querying the past two weeks of query history. If you need to query more history, for example in an initial load, consider using the S3 miner first. After the initial load, you can modify the miner's configuration to use query history extraction.
- (Optional) For Advanced Config, keep Default for the default configuration or click Advanced to configure the miner:
- For Cross Connection, click Yes to extract lineage across all available data source connections or click No to only extract lineage from the selected Teradata connection.
- For Control Config, if Atlan support has provided you with a custom control configuration, select Custom and enter the configuration into the Custom Config box. You can also:
- Enter
{“ignore-all-case”: true}to enable crawling assets with case-sensitive identifiers.
- Enter
🚨 Careful! If running the miner for the first time, Atlan recommends setting a start date roughly three days prior to the current date and then scheduling it daily to build up to two weeks of query history. Mining two weeks of query history on the first miner run may cause delays. Atlan requires a minimum lag of 24 to 48 hours to capture all the relevant transformations that were part of a session. Learn more about the miner logic here.
Run the miner
To run the Teradata miner, after completing the steps above:
- To run the miner once, immediately, at the bottom of the screen, click the Run button.
- To schedule the miner to run hourly, daily, weekly, or monthly, at the bottom of the screen, click the Schedule & Run button.
Once the miner has completed running, you will see lineage for Teradata assets that were created in Teradata between the start date and when the miner ran! 🎉