Securely Interconnecting EC2 Instances Across Private Amazon VPC Subnets Using Netmaker
Overview
Amazon Virtual Private Cloud (VPC) lets you provision a logically isolated networking environment with full control. A VPC contains one or more subnets, which are IP address ranges. Subnets can be public or private:
Public Subnets have a direct route to an Internet Gateway. Resources in these subnets can access the internet directly.
Private Subnets lack a direct internet route and require a NAT device (NAT Gateway or NAT Instance) to initiate outbound internet connections.
AWS provides two NAT options:
NAT Gateway (fully managed by Amazon): Allows private subnet resources to access the internet, but does not support port forwarding.
NAT Instance: A self-managed EC2 instance that supports port forwarding.
Additionally, Security Groups act as virtual firewalls, controlling inbound and outbound traffic for associated resources.
Goal
Connecting devices behind NAT or restrictive firewalls can sometimes be tricky. This guide helps you securely interconnect EC2 instances within private subnets across different VPCs—whether or not those instances have Netclient installed. It covers both Netmaker Pro and Community Edition scenarios.
Prerequisites
Netclient uses UDP port 443 by default to communicate with other Netclients. This port must be allowed in your security group. Additionally, Netclient uses TCP port 51821 by default for endpoint detection and metrics collection. Although not required, we recommend allowing this port in your security group as well.
For more information about the advantages of the endpoint detection feature, see: https://help.netmaker.io/en/articles/10026438-what-is-endpoint-detection
If a firewall is running on your EC2 instances, you’ll also need to allow these ports on each instance.
It’s also a good idea to temporarily enable ICMP traffic for testing purposes.
Advantages of Netmaker Pro
Netmaker Pro is fully capable of interconnecting hard-to-reach edge devices with minimal intervention. It allows you to designate a publicly reachable device—such as the Netmaker Server—as a Failover Node. With this feature, Netmaker detects if peers are unable to communicate directly (for example, EC2 instances in different private subnets behind NAT). Netmaker will automatically reroute traffic through the Failover Node, enabling those peers to connect via that node.
See: https://docs.netmaker.io/docs/features/failover-servers-pro
You can only designate one device as a Failover Node per VPN network. Note that this feature only works if the EC2 instances are running Netclient.
For larger deployments, Netmaker Pro also provides a [Relay] Gateway feature. You can assign multiple Relay Gateways and control which nodes they relay traffic for. A Relay Gateway can be any public machine (or an EC2 instance running Netclient in a public subnet) which relays traffic to EC2 instances in a private subnet within the same VPC.
See: https://docs.netmaker.io/docs/features/gateways#relay-configuration
If using a Failover Node or Relay Gateway is not an option—such as when using the Community Edition of Netmaker, or when aiming to minimize traffic hops—then additional manual configuration on AWS and on the devices is needed.
For Community Edition Users
Note: Relay Gateway becomes available in Community Edition starting version 0.90.0.
To enable EC2 instances in different private subnets to communicate with each other, manual configurations—such as setting port forwarding and routing rules—are necessary.
As noted earlier, port forwarding can only be performed on a NAT Instance, not on a NAT Gateway. The sections below first cover NAT Instance setups, then NAT Gateway scenarios.
First, plan the UDP port range you'll use. If you have multiple EC2 instances behind a specific NAT Instance, each must have its own unique port number. The dynamic/private port range 49152–65535 is suitable. If you have multiple private subnets, you may reuse the same port range across them (for example, 51821–52821), though it's recommended to use different port numbers per subnet due to observed AWS quirks.
Create a security group allowing the full chosen port range. This range must be allowed both on the public subnet (where the NAT Instance resides) and on the private subnet security groups.
On your NAT Instance, add port forwarding rules using iptables. Example for forwarding UDP port 51821 to a target instance at 192.168.0.140:
You’ll need one corresponding rule for each EC2 instance running Netclient behind the NAT Instance.
Review configured rules:
Delete a rule by number:
Once everything looks good, save the rules using iptables-save/iptables-restore or the iptables-persistent package.
Your rules should look similar to the following screenshot:
Then, in the NMUI, go to the Devices page and set each device's port to static using its corresponding port number.
After these steps, EC2 instances behind the NAT Instance should be able to securely communicate with other nodes in the Netmaker network via the encrypted tunnel.
Securely Interconnecting EC2 Instances Behind a NAT Gateway
This scenario is similar to a full-mesh site-to-site setup but specific to AWS.
In this setup, you do not need to install Netclient (or anything at all) on EC2 instances in the private subnet behind a NAT Gateway. Instead, run Netclient on another EC2 instance in the public subnet (same VPC) to act as a secure tunnel gateway into your Netmaker network.
Important: AWS does not route typical private subnet traffic to the public subnet by default. Therefore, your Netmaker network must use the Shared Address Space: 100.64.0.0 – 100.127.255.255 (recommended).
Once the public EC2 instance with Netclient has joined your Netmaker network, configure it as an Egress Gateway, exposing either the entire private subnet or specific hosts. Example subnets:
Public subnet: 192.168.56.0/28
Private subnet: 192.168.56.128/28
You can expose the entire private subnet (192.168.56.128/28) or individual hosts (e.g., 192.168.56.130/32, 192.168.56.135/32).
You may enable or disable NAT for egress traffic:
If NAT is enabled, source addresses of traffic entering the private subnet will appear as the Egress Gateway's Netmaker IP.
If NAT is disabled, the source address will be the requesting node's IP from within the Netmaker network.
If NAT is enabled, nodes in your Netmaker network can initiate and reach EC2 instances in the private subnet—but the private subnet hosts cannot initiate to the Netmaker network unless you add routes (see below).
To allow two-way communication, add a route in your VPC so that traffic from the private subnet knows how to reach the Netmaker network via the Egress Gateway. For example, if your Netmaker network is 100.110.0.0/24, add a route pointing that prefix to the Egress Gateway instance.
Additionally, because the Egress Gateway instance is not always the source or destination of the traffic it handles, disable source/destination checks on it. To do this:
Select the EC2 instance in the AWS console.
Go to Actions > Networking > Change source/destination check.
Tick the Stop checkbox and save.
After adding the appropriate VPC routes and disabling source/destination checks, two-way communication should be possible between EC2 instances in the private subnet and nodes in your Netmaker network.
Note: Route management is manual. If new egress ranges are added, you must manually add a VPC routing entry for each new range.
Was this helpful?