Hacking the VPC: ELB as a Bastion

a common deployment structure for amazon virtual private clouds (vpcs) is to separate your servers into public and private subnets. for example, you put your webservers into the public subnet, and database servers in the private subnet. or for more security you put all of your servers in the private subnet, with an elastic load balancer (elb) in the public subnet as the only point-of-contact with the open internet.

the problem with this second architecture is that you have no way to get to those servers for troubleshooting: the definition of a private subnet is that it does not expose servers to the internet.

the standard solution involves a “bastion” host: a separate ec2 instance that runs on the public subnet and exposes a limited number of ports to the outside world. for a linux-centric distribution, it might expose port 22 (ssh), usually restricted to a limited number of source ip addresses. in order to access a host on the private network, you first connect to the bastion host and then from there connect to the private host (although there's a neat trick with netcat that lets you connect via the bastion without an explicit login).

the problem with a bastion host — or, for windows users, an rd gateway — is that it costs money. not much, to be sure: ssh forwarding doesn't require much in the way of resources, so a t2.nano instance is sufficient. but still …

it turns out that you've already got a bastion host in your public subnet: the elb. you might think of your elb as just front-end for your webservers: it accepts requests and forwards them to one of a fleet of servers. if you get fancy, maybe you enable session stickiness or do https termination at the load balancer. but what you may not realize is that an elb can forward any tcp port.

so, let's say that you're running some windows servers in the private subnet. to expose them to the internet, go into your elb config and forward traffic from port 3389:

of course, you don't really want to expose those servers to the internet, you want to expose them to your office network. that's controlled by the security group that's attached to the elb; add an inbound rule that just allows access from your home/office network (yeah, i'm not showing my real ip here):

lastly, if you use an explicit security group to control traffic from the elb to the servers, you'll also need to open the port on it. personally, i like the idea of a “default” security group that allows all components of an application within the vpc to talk with each other.

you should now be able to fire up your favorite rdesktop client and connect to a server.

> xfreerdp --plugin cliprdr -u administrator 52.15.40.131
loading plugin cliprdr
connected to 52.15.40.131:3389
password: 
...

the big drawback, of course, is that you have to control over which server you connect to. but for many troubleshooting tasks, that doesn't matter: any server in the load balancer's list will show the same behavior. and in development, where you often have only one server, this technique lets you avoid creating special configuration that won't run in production.

actually, the definition of a public subnet is that it routes non-vpc traffic to an internet gateway, which is a precondition for exposing servers to the internet. however, this isn't a sufficient condition: even if you have an internet gateway you can prevent access to a host by not giving it a public ip. but such pedantic distinctions are not really relevant to the point of this post; for practical purposes, a private subnet doesn't allow any access from the internet to its hosts, while a public subnet might.

i should clarify: the classic load balancer can forward any port; an application load balancer just handles http and https, but has highly configurable routing. see the docs for more details.