How to set up a private network link to Amazon Athena

AWS PrivateLink creates a secure, private connection between services running in AWS. This document describes the steps to set this up between Amazon Athena and Atlan.

Request Atlan's details

Before configuring the connection, you will need the following:

• VPC endpoint ID of the Atlan VPC endpoint in the following format — vpce-0d90d77d1be568544. This will be required to create the IAM policy.
• To enter a hostname for crawling Amazon Athena:
  • If private DNS hostnames are enabled, enter the default Athena endpoint in the following format — https://athena.<region>.amazonaws.com — and it will resolve to your VPC endpoint.
  • If private DNS hostnames are not enabled, enter the primary DNS name of the Atlan VPC endpoint in the following format — vpce-<hash>-<hash.>vpce-svc-<hash>.<region>.vpce.amazonaws.com — as retrieved from Atlan support.

Request it from Atlan support.

Create IAM policy

To create an IAM policy with the necessary permissions, follow the steps in the AWS Identity and Access Management User Guide.

Create the policy using the following JSON:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAthenaListDataCatalog",
            "Effect": "Allow",
            "Action": [
                "athena:ListDataCatalogs"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:SourceVpce": [
                        "<vpce-endpoint-id>"
                    ]
                }
            }
        },
        {
            "Sid": "AllowAthenaActions",
            "Effect": "Allow",
            "Action": [
                "athena:StartQueryExecution",
                "athena:GetQueryResults",
                "athena:DeleteNamedQuery",
                "athena:GetNamedQuery",
                "athena:ListQueryExecutions",
                "athena:StopQueryExecution",
                "athena:GetQueryResultsStream",
                "athena:ListNamedQueries",
                "athena:CreateNamedQuery",
                "athena:GetQueryExecution",
                "athena:BatchGetNamedQuery",
                "athena:BatchGetQueryExecution",
                "athena:GetWorkGroup",
                "athena:GetTableMetadata",
                "athena:GetDatabase",
                "athena:GetDataCatalog",
                "athena:ListDatabases",
                "athena:ListTableMetadata"
            ],
            "Resource": [
                "arn:aws:athena:us-east-2:666568140392:datacatalog/*",
                "arn:aws:athena:us-east-2:666568140392:workgroup/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:SourceVpce": [
                        "<vpce-endpoint-id>"
                    ]
                }
            }
        },
        {
            "Sid": "AllowGlueActionsViaAthena",
            "Effect": "Allow",
            "Action": [
                "glue:GetDatabase",
                "glue:GetDatabases",
                "glue:CreateDatabase",
                "glue:GetTables",
                "glue:GetTable",
                "glue:SearchTables",
                "glue:GetTableVersions",
                "glue:GetTableVersion",
                "glue:GetPartition",
                "glue:GetPartitions",
                "glue:GetUserDefinedFunctions",
                "glue:GetUserDefinedFunction"
            ],
            "Resource": [
                "arn:aws:glue:us-east-2:666568140392:tableVersion/*/*/*",
                "arn:aws:glue:us-east-2:666568140392:catalog",
                "arn:aws:glue:us-east-2:666568140392:table/*/*",
                "arn:aws:glue:us-east-2:666568140392:database/*"
            ],
            "Condition": {
                "ForAnyValue:StringEquals": {
                    "aws:CalledVia": [
                        "athena.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Sid": "AllowS3ActionsOnDataViaAthena",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::demo-wide-world-importers",
                "arn:aws:s3:::demo-wide-world-importers/*"
            ],
            "Condition": {
                "ForAnyValue:StringEquals": {
                    "aws:CalledVia": [
                        "athena.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Sid": "AllowS3ActionsOnMetadataViaAthena",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts",
                "s3:AbortMultipartUpload",
                "s3:CreateBucket",
                "s3:PutObject"
            ],
            "Resource": [
                "arn:aws:s3:::source-curation-athena-metadata",
                "arn:aws:s3:::source-curation-athena-metadata/*"
            ],
            "Condition": {
                "ForAnyValue:StringEquals": {
                    "aws:CalledVia": [
                        "athena.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

• Replace <vpce-endpoint-id> with the VPC endpoint ID received from Atlan support.

Create an IAM user

Create an AWS IAM user and attach the policy created above to this user.

To create an AWS IAM user:

1. Follow the steps in the AWS Identity and Access Management User Guide.
2. On the Set permissions page, attach the policy created in the previous step to this user.
3. Refer to managing access keys for IAM users to create an access key for the new user.
4. Once the user is created, view or download the user's access key ID and secret access key.

The connection is now established. You can now use the DNS name of the Atlan VPC endpoint as the hostname to crawl Amazon Athena in Atlan! 🎉