How do I disable cloning Looker projects from git?
The looker-extractor tool clones your Looker git project to parse field-level lineage. By default, the extractor needs the looker_git_private_key
and looker_git_private_key_passphrase
secrets. To disable this behavior, you need to set the USE_FIELD_LEVEL_LINEAGE
parameter to "false"
.
For example:
services:
# Example Looker connection
looker-example:
<<: *extract
environment:
<<: *looker-defaults
INCLUDE_PROJECTS: "project-1,project-2"
USE_FIELD_LEVEL_LINEAGE: "false"
volumes:
- ./output/looker-example:/output/process
secrets:
- looker_config
secrets:
looker_config:
file: ./looker.ini
Note that we refer to only the looker-config
secret in this looker-example
service. This is possible because the USE_FIELD_LEVEL_LINEAGE
environment variable is set to "false"
.
How do I use different credentials for different connections?
You may define as many Looker connections under the services
section as you want. As mentioned before, by default each connection requires the following secrets:
looker_config
looker_git_private_key
looker_git_private_key_passphrase
You may want to define many connections with different credentials. For example, you may want to extract field-level lineage for some projects and not for others. To do this, you need to define and refer to those secrets separately for each connection.
For example:
services:
# The first-connection uses secrets with default names,
# so no need to specify its own `secrets` section
first-connection:
<<: *extract
environment:
<<: *looker-defaults
INCLUDE_PROJECTS: "project-1"
USE_FIELD_LEVEL_LINEAGE: "true"
volumes:
- ./output/first-connection:/output/process
# The second-connection uses alternative secrets,
# so it defines its own `secrets` section to refer to them
second-connection:
<<: *extract
environment:
<<: *looker-defaults
INCLUDE_PROJECTS: "project-2"
USE_FIELD_LEVEL_LINEAGE: "true"
volumes:
- ./output/first-connection:/output/process
secrets:
- target: looker_config
source: alternative_config
- target: looker_git_private_key
source: alternative_private_key
- target: looker_git_private_key_passphrase
source: alternative_key_passphrase
secrets:
# Credentials for the first-connection (default names)
looker_config:
file: ./looker.ini
looker_git_private_key:
file: ./id_rsa
looker_git_private_key_passphrase:
file: ./passphrase.txt
# Credentials for the second-connection
alternative_config:
file: ./path/to/alternative/looker.ini
alternative_private_key:
file: ./path/to/alternative/id_rsa
alternative_key_passphrase:
file: ./path/to/alternative/passphrase.txt
In the second-connection
secrets list, note that:
- the
target
attribute gives the default name of the secret the extractor expects - the
source
attribute gives the alternative secret name