Once you have set up the thoughtspot-extractor tool, you can extract metadata from your on-premises ThoughtSpot instances by completing the following steps.
Run thoughtspot-extractor
Crawl all ThoughtSpot connections
To crawl all ThoughtSpot connections using the thoughtspot-extractor tool:
- Log into the server with Docker Compose installed.
- Change to the directory containing the compose file.
- Run Docker Compose:
sudo docker-compose up
Crawl a specific connection
To crawl a specific ThoughtSpot connection using the thoughtspot-extractor tool:
- Log into the server with Docker Compose installed.
- Change to the directory containing the compose file.
- Run Docker Compose:
sudo docker-compose up <connection-name>
(Replace <connection-name> with the name of the connection from the services section of the compose file.)
(Optional) Review generated files
The thoughtspot-extractor tool will generate many folders with JSON files for each service. For example:
tagsanswersliveboardsanswer-sql-queriesliveboard-sql-queries
You can inspect the metadata and make sure it is acceptable for providing metadata to Atlan.
Upload generated files to S3
To provide Atlan access to the extracted metadata, you will need to upload the metadata to an S3 bucket.
To upload the metadata to S3:
- Ensure that all files for a particular connection have the same prefix. For example,
output/thoughtspot-example/filter/answers/result-0.json,output/thoughtspot-example/filter/liveboards/result-0.json,output/thoughtspot-example/filter/answer-sql-queries/result-0.json, and so on. - Upload the files to the S3 bucket using your preferred method.
For example, to upload all files using the AWS CLI:
aws s3 cp output/thoughtspot-example/filter s3://my-bucket/metadata/thoughtspot-example --recursiveCrawl metadata in Atlan
Once you have extracted metadata on-premises and uploaded the results to S3, you can crawl the metadata into Atlan:
Be sure to select Offline for the Extraction method.