How to set up a private network link to Microsoft SQL Server on Amazon EC2

AWS PrivateLink creates a secure, private connection between services running in AWS. This document describes the steps to set this up between Microsoft SQL Server on Amazon EC2 and Atlan.

Prerequisites

You should already have the following:

• Your own non-default VPC configured in AWS.
• A Microsoft SQL Server on Amazon EC2 instance running in AWS, linked to the non-default VPC.
• Private subnets defined within the non-default VPC sufficient for availability.

Create security group

You will need to create a security group for the following:

• Microsoft SQL Server on Amazon EC2 instance
• Network Load Balancer (NLB)

Microsoft SQL Server on Amazon EC2 instance

You can either create a new security group or add the following rule to an existing security group already attached to your Microsoft SQL Server on Amazon EC2 instance.

To create a security group for your Microsoft SQL Server on Amazon EC2 instance:

1. Open the Amazon VPC console.
2. From the left menu, under Security, click Security Groups.
3. Click the Create security group button.
4. Enter a name and description for the new security group.
5. From the VPC list, select the VPC where your Microsoft SQL Server on Amazon EC2 instance is located.
6. For Inbound rules, leave this blank until after you have created a security group for the Network Load Balancer. Return to this step once you have created it, click the Add rule button, and then add the following rule:
   
   1. For Type, use MSSQL if you are using the default port (1433), or use Custom and enter your port under Port range.
   2. For Destination, add the security group you created for the NLB.
7. Click Create security group to finish setup.

Network Load Balancer

To create a security group for the Network Load Balancer:

1. Open the Amazon VPC console.
2. From the left menu, under Security, click Security Groups.
3. Click the Create security group button.
4. Enter a name and description for the new security group.
5. From the VPC list, select the VPC where your Microsoft SQL Server on Amazon EC2 instance is located.
6. For Outbound rules, click the Add rule button and then add the following rule:
   
   1. For Type, use MSSQL if you are using the default port (1433), or use Custom and enter your port under Port range.
   2. For Destination, add the security group you created for your Microsoft SQL Server on Amazon EC2 instance.
   3. Click Save.
7. Click Create security group to finish setup.

Create a target group

To create a target group for the NLB:

1. Open the Amazon EC2 console.
2. From the left menu, under Load Balancing, click Target Groups.
3. Click Create target group.
4. For Basic configuration, enter the following details:
   1. For Choose a target type, keep Instances.
   2. For Target group name, enter a unique name for the new target group.
   3. For Protocol, select TCP.
   4. For Port, enter 1433.
   5. For IP address type, select IPv4.
   6. For VPC, select the VPC where your Microsoft SQL Server on Amazon EC2 instance is located.
5. In the Health checks section, change the protocol to TCP and keep Advanced health check settings as the default.
6. Click Next to proceed.
7. To register your Amazon EC2 instance, on the Register targets page:
   1. For Available instances, select your Amazon EC2 instance running Microsoft SQL Server.
   2. Keep the default port 1433 and then choose Include as pending below.
8. At the bottom of the form, click the Create target group button.

Create internal Network Load Balancer

To create an NLB:

1. Open the Amazon EC2 console.
2. From the left menu, under Load Balancing, click Load Balancers.
3. At the top of the screen, click the Create Load Balancer button.
4. Under the Network Load Balancer option, click the Create button.
5. Enter the following Basic configuration settings for the load balancer:
   1. For Load balancer name, enter a unique name.
   2. For Scheme, select Internal.
   3. For IP address type, select IPv4.
6. Enter the following Network mapping settings for the load balancer:
   1. For VPC, select the VPC where your Microsoft SQL Server on Amazon EC2 instance is located.
   2. For Mappings, select the availability zones with private subnets.
7. For Security groups, select the security group you created for the Network Load Balancer.
   💪 Did you know? The Enforce inbound rules on PrivateLink traffic setting is turned on by default and cannot be modified until after the load balancer has been created. If this setting is left on, you will need to contact Atlan support and request the CIDR range of Atlan's cluster to add as an inbound rule on the NLB security group. To turn it off, follow these instructions.
8. Enter the following Listeners and routing settings for the load balancer:
   1. For Protocol, select TCP.
   2. For Port, enter 1433.
   3. For Target group, select the target group you created.
9. Review your configuration, and click Create load balancer.

Verify target group is healthy

To verify that the target group is healthy:

1. From the EC2 menu on the left, under Load Balancing, click Target Groups.
2. From the Target groups table, click the link to the target group you created above.
3. At the bottom of the screen, under the Details tab, check that there is a 1 under both Total targets and Healthy. (Note: This number could be more than 1 if you have a multi-node deployment.)

Create endpoint service

To create an endpoint service:

1. Open the Amazon VPC console.
2. From the left menu, under Virtual private cloud, click Endpoint services.
3. At the top of the page, click the Create endpoint service button.
4. Enter the following Endpoint service settings:
   1. For Name, enter a meaningful name.
   2. For Load balancer type, choose Network.
5. For Available load balancers, select the load balancer you created above.
6. Enter the following Additional settings:
   1. For Require acceptance for endpoint, enable Acceptance required to require manual acceptance of connection requests to your endpoint service. Otherwise, these requests will be accepted automatically.
   2. For Enable private DNS name, leave unchecked.
   3. For Supported IP address types, enable IPv4.
7. At the bottom of the form, click the Create button.
8. Once the endpoint service has been created, navigate to the Details page. From the Details page:
   1. Under Service Name, copy the value to send to Atlan.
   2. Under Availability Zones, copy the zones to send to Atlan.

Allow Atlan account access

To allow Atlan's account access to the service, from within the endpoint service screen:

1. At the bottom of the screen, change to the Allow principals tab.
2. At the top of the Allow principals table, click the Allow principals button.
3. Under Principals to add and ARN, enter the Atlan account ID and root principal — for example, arn:aws:iam::<account_id>:root.
4. At the bottom of the form, click the Allow principals button.

Notify Atlan support

Once all of the above steps have been completed, contact Atlan support and provide the following details:

• Service name of the endpoint service
• Availability zones for the endpoint service

There are additional steps Atlan then needs to complete:

• Creating a security group.
• Creating an endpoint.

Once the Atlan team has confirmed the configuration is ready, please continue with the remaining steps.

Accept the consumer connection request

To accept the consumer connection request, from within AWS:

1. Navigate to Services, then Networking & Content Delivery, then VPC.
2. From the menu on the left, under Virtual private cloud, click Endpoint services.
3. From the Endpoint services table, select the endpoint service you created in Create endpoint service.
4. At the bottom of the screen, change to the Endpoint connections tab.
   1. You should see a row in the Endpoint connections table with a State of Pending acceptance.
   2. Select this row, and click the Actions button and then Accept endpoint connection request.
5. Wait for this to complete, it could take about 30 seconds.

Request DNS name from Atlan

Contact Atlan support to request the regional DNS name of the VPC endpoint that Atlan created in the following format — vpce-<hash>-<hash.>vpce-svc-<hash>.<region>.vpce.amazonaws.com. This is the hostname you will need to use to connect to your Microsoft SQL Server on Amazon EC2 instance from within Atlan.

😅 The connection is now established. You can now use the DNS name of the Atlan VPC endpoint as the hostname to crawl Microsoft SQL Server in Atlan! 🎉