How to crawl on-premises Looker

Once you have set up the looker-extractor tool, you can extract metadata from your on-premises Looker instances using the following steps.

Run looker-extractor

Crawl all Looker connections

To crawl all Looker connections using the looker-extractor tool:

1. Log into the server with Docker Compose installed.
2. Change to the directory containing the compose file.
3. Run Docker Compose: sudo docker-compose up

Crawl a specific connection

To crawl a specific Looker connection using the looker-extractor tool:

1. Log into the server with Docker Compose installed.
2. Change to the directory containing the compose file.
3. Run Docker Compose: sudo docker-compose up <CONNECTION-NAME>

(Replace <CONNECTION-NAME> with the name of the connection from the services section of the compose file.)

(Optional) Review generated files

The looker-extractor tool will generate many JSON files for each service. For example:

• projects.json
• dashboards.json
• dashboard_tiles.json
• looks.json
• and many others

You can inspect the metadata and make sure it is acceptable to provide the metadata to Atlan.

Upload generated files to S3

To provide Atlan access to the extracted metadata you will need to upload the metadata to an S3 bucket.

To upload the metadata to S3:

1. Ensure all files for a particular database have the same prefix. For example, metadata/looker/projects.json, metadata/looker/dashboards.json, etc.
2. Upload the files to the S3 bucket using your preferred method.

For example, to upload all files using the AWS CLI:

aws s3 cp output/looker-example s3://my-bucket/metadata/looker-example --recursive

Crawl metadata in Atlan

Once you have extracted metadata on-premises and uploaded the results to S3, you can crawl the metadata into Atlan:

• How to crawl Looker

Be sure you select Offline for the Extraction method.