Connecting to an RDS in a Private VPC

Overview

There are following scenarios for accessing an RDS DB instance by the Matillion Instance.

• A DB Instance in a VPC Accessed by Matillion Instance in the Same VPC.


Same VPC

Important Information

The simplest way to manage access between Matillion instances and RDS DB instances in the same VPC is to do the following:

• Create a VPC security group for RDS DB instances to be in. This security group can be used to restrict access to the RDS DB instances.
• Create a VPC security group for Matillion instances to be in. This security group can, if needed, allow access to the Matillion instance from the internet by using the VPC's routing table.
• Create custom rules in the security group for RDS DB instances that allow connections from the security group you created for the Matillion instances. This would allow any member of the security group to access the DB instances.
• A DB Instance in a VPC Accessed by Matillion Instance in a Different VPC

Different VPC

When your DB instance is in a different VPC from the Matillion instance , you can use VPC peering to access the DB instance.

A VPC peering connection is a networking connection between two VPCs that enables you to route traffic between them using private IP addresses. Instances in either VPC can communicate with each other as if they are within the same network. You can create a VPC peering connection between your own VPCs, with a VPC in another AWS account, or with a VPC in a different AWS Region.

This guide looks at how to set up VPC Peering to allow Matillion instance available in one VPC to use VPC Peer connection to access the data from the another VPC (Amazon RDS Databse). The concept is the same for any data held within the VPC.

For security reasons, RDS database might be held in a Private Subnet of the VPC that doesn't allow public access. Whereas, Matillion instance, which might be in a Public Subnet of the another VPC, can still connect to this RDS database using VPC Peering.

The following diagram depicts what final architecture will look like if Matillion Instance and RDS Database placed in different VPC.

VPC Peering Architecture

Create a VPC

Navigate to the Amazon VPC Dashboard in the AWS Management Console. Click Create VPC and fill in all the details required to create a VPC.

Create VPC

Create and Attach Internet Gateway

On the Internet Gateways tab of the VPC Dashboard, create a new Internet gateway. Attach the gateway to the newly created VPC.

Internet Gateway

Create a Custom Route Tables

On the Route Tables tab of the VPC Dashboard, click Create route table.

Create Route Tables

Add a route to the route table for the Internet gateway by clicking Edit routes tab.

Add Route Tables

Create Public and Private Subnet

Create a subnet in the required availability zone by clicking Create subnet

Create subnet

Change the route table for the previously created subnet from the main route table to the custom route table, if needed. And , click Edit route table association

Add Subnet

Create Security Group

Create a Security Group for the Matillion instance to be provisioned in the public subnet.

Create Security Group

Authorize inbound traffic from your local IP address. The default outbound rules should be fine.

Add Inbound Rules

Similary, create a security group for the RDS instance to be provisioned in the private subnet.

Peering Connection

A Peering Connection is required to route traffic between two VPCs. These VPCs can be in the same AWS account, or in different accounts if required.

Create VPC Peering Connection

To create Peering connection, navigate to Peering Connection to the left of the menu and click Create peering connection.

Create Peering Connection

Accept VPC Peering Connection

This peering request will appear in Peering Connections and now needs to be accepted by the account the VPC to peer with is in:

Peering Connection Created

VPC Peering Connection Status

Check that the status is now active on both VPCs.

Peering Connection Status

Some additional steps to complete the connection

Once the Peering Connection completed but still we need to follow some additional steps to complete the connectivity between two VPCs.

Create Routes on both the ends.

On VPC 1 where Matillion resides, we need to provide CIDR for destination VPC (VPC 2) - where RDS resides, and target should be "Peering Connection". Click on Save routes.

Edit Routes

For the VPC that Matillion is in, all traffic to the IP associated with the Private VPC needs to be directed to the Peering Connection:

Set Route Table for Matillion

For the RDS VPC, all traffic to the Matillion instance needs to be directed to use the Peering Connection. This was done by directing all other traffic to the Peering Connection but this IP range could be limited:

Set route Table for RDS

Matillion now has a private route to the RDS instance without having to use the internet. The RDS Query and RDS Bulk Load components will now work as expected from the Matillion instance.