How to set up Amazon Redshift

Atlan supports fetching metadata from Amazon Redshift for the following types of deployment:

• Provisioned
• Serverless

Grant permissions

For all supported authentication mechanisms except IAM role authentication on serverless deployment, you must first grant the following permissions on Amazon Redshift. For IAM role authentication on serverless deployment only, skip to this step.

Create a group and user

To create a group and user, run the following commands:

CREATE GROUP atlan_users;
CREATE USER atlan_user password '<pass>' IN GROUP atlan_users;

• Replace <pass> with the password for the atlan_user being created.
• To crawl Amazon Redshift, for Username, you must enter the username you configured for the database user — for example, atlan_user.

Grant required permissions to group

To grant the minimum required permissions, run the following commands:

GRANT temp ON DATABASE <database_name> TO GROUP atlan_users;
GRANT usage ON schema <schema_name> TO GROUP atlan_users;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO GROUP atlan_users;
GRANT SELECT ON pg_catalog.svv_table_info TO GROUP atlan_users;
GRANT SELECT ON pg_catalog.svv_external_tables TO GROUP atlan_users;
GRANT SELECT ON pg_catalog.svv_redshift_databases TO GROUP atlan_users;
GRANT SELECT ON pg_catalog.svv_all_schemas TO GROUP atlan_users;
GRANT SELECT ON pg_catalog.svv_all_columns TO GROUP atlan_users;

• Replace <database_name> with the name of the database to crawl.
• Replace <schema_name> with the name of the schema to crawl.

The permissions are used for the following:

• SVV_TABLE_INFO is used to obtain information on the table ID to table/schema/database relation.
• SVV_EXTERNAL_TABLES is used to obtain information on external tables.

(Optional) Grant permissions for role-based authentication on serverless

For IAM role-based authentication on Amazon Redshift serverless deployment only, you must first grant the following permissions on Amazon Redshift.

Create a role

To create a role, run the following commands:

CREATE ROLE atlan_role;

Grant required permissions to role

To grant the minimum required permissions, run the following commands:

GRANT temp ON DATABASE <database_name> TO ROLE atlan_role;
GRANT usage ON schema <schema_name> TO ROLE atlan_role;
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO ROLE atlan_role;
GRANT SELECT ON pg_catalog.svv_table_info TO ROLE atlan_role;
GRANT SELECT ON pg_catalog.svv_external_tables TO ROLE atlan_role;
GRANT SELECT ON pg_catalog.svv_redshift_databases TO ROLE atlan_role;
GRANT SELECT ON pg_catalog.svv_all_schemas TO ROLE atlan_role;
GRANT SELECT ON pg_catalog.svv_all_columns TO ROLE atlan_role;

• Replace <database_name> with the name of the database to crawl.
• Replace <schema_name> with the name of the schema to crawl.

The permissions are used for the following:

• SVV_TABLE_INFO is used to obtain information on the table ID to table/schema/database relation.
• SVV_EXTERNAL_TABLES is used to obtain information on external tables.

Grant additional permissions for mining query history

To grant the additional permissions needed to mine query history, run the following commands:

GRANT SELECT on pg_catalog.stl_ddltext to GROUP atlan_users;
GRANT SELECT on pg_catalog.stl_query to GROUP atlan_users;
GRANT SELECT on pg_catalog.stl_connection_log to GROUP atlan_users;
GRANT SELECT on pg_catalog.stl_undone to GROUP atlan_users;
GRANT SELECT on pg_catalog.stl_insert to GROUP atlan_users;
GRANT SELECT on pg_catalog.svl_statementtext to GROUP atlan_users;
ALTER USER atlan_user SYSLOG ACCESS UNRESTRICTED;

The additional permissions are used for the following:

• STL_DDLTEXT is used for DDL queries.
• STL_QUERY is used for DML and regular queries.
• STL_CONNECTION_LOG is used to obtain the session ID that a query is part of.
• STL_UNDONE is used to obtain information about transactions that have been undone or rolled back.
• STL_INSERT is used to obtain the table ID used in the insert queries.
• SVL_STATEMENTTEXT is used to obtain the query text for all queries.
• SYSLOG ACCESS UNRESTRICTED is used to access all queries performed by any user in the system tables above.

To use basic authentication, your setup is now complete. To configure IAM-based authentication, you need to continue with the following steps.

(Optional) Create IAM policy

All IAM-based authentication mechanisms require an IAM policy to be created. For all supported authentication mechanisms except IAM role authentication on serverless deployment, create the following IAM policy. For IAM role authentication on serverless deployment only, skip to this step.

To create an IAM policy with the necessary permissions follow the steps in the AWS Identity and Access Management User Guide.

Create the policy using the following JSON:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "redshift:GetClusterCredentials"
      ],
      "Resource": [
        "arn:aws:redshift:<region>:<account_id>:dbuser:<redshift_cluster_identifier>/atlan_user",
        "arn:aws:redshift:<region>:<account_id>:dbname:<redshift_cluster_identifier>/<database>"
      ]
    }
  ]
}

• Replace <region> with the AWS region of your Redshift instance.
• Replace <account_id> with your account ID.
• Replace <redshift_cluster_identifier> with your Redshift cluster identifier.
• Replace <database> with the name of the Redshift database.

(Optional) Create IAM policy for role-based authentication on serverless

For IAM role-based authentication on Amazon Redshift serverless deployment only, create an IAM policy with the necessary permissions. Follow the steps in the AWS Identity and Access Management User Guide.

1. Create the policy using the following JSON:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "redshift-serverless:GetCredentials"
         ],
         "Resource": [
           "arn:aws:redshift-serverless:<region>:<account_id>:workgroup/<workgroup_identifier>",
         ]
       }
     ]
   }

   • Replace <region> with the AWS region of your Amazon Redshift instance.
   • Replace <account_id> with your AWS account ID.
   • Replace <workgroup_identifier> with your Amazon Redshift serverless workgroup identifier.
2. Configure tag for IAM role:

   {
     RedshiftDbRoles : <role>
   }

   • Replace <role> with the role you created.

(Optional) Choose IAM-based authentication mechanism

Using the policy created above, configure one of the following options for authentication.

User-based authentication

To configure user-based authentication:

1. Create an AWS IAM user by following the steps in the AWS Identity and Access Management User Guide.
2. On the Set permissions page, attach the policy created in the previous step to this user.
3. Once the user is created, view or download the user's access key ID and secret access key.
   🚨 Careful! This will be your only opportunity to view or download the access keys. You will not have access to them again after leaving the user creation screen.

Role delegation-based authentication

To configure role delegation-based authentication:

1. Raise a support ticket to get the ARN of the Node Instance Role for your Atlan EKS cluster.
2. Create a new role in your AWS account by following the steps in the AWS Identity and Access Management User Guide.
   1. When prompted for policies, attach the policy created in the previous step to this role.
   2. When prompted, create a trust relationship for the role using the following trust policy. (Replace <atlan_nodeinstance_role_arn> with the ARN received from Atlan support.)

      {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "AWS": "<atlan_nodeinstance_role_arn>"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
          }
        ]
      }

3. (Optional) To use an external ID for additional security, paste the external ID into the policy:
   

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": {
           "AWS": "<atlan_nodeinstance_role_arn>"
         },
         "Action": "sts:AssumeRole",
         "Condition": {
           "StringEquals": {
             "sts:ExternalId": "<atlan_external_id>"
           }
         }
       }
     ]
   }

   • Replace <atlan_external_id> with the external ID you want to use.
4. Now, reach out to Atlan support with:
   • The name of the role you created above.
   • The ID of the AWS account where the role was created.