ec2 hostnames, dns and you

04/04/2014

When we first started out in this kooky cloud world, we quickly learned one of the fundamental shifts that the cloud represents: server naming. For folks crossing over for their first foray into AWS or other cloud systems, the lack of conventional naming approaches, and the utter lack of network planning can be bewildering. Add to the confusion, some genuinely good suggestions that you flat abandon server naming entirely and you're left with more questions than answers as you plan your cloud migration.

We decided that, while really into this cloud thing, we still wanted individual server hostnames. At a small scale naming can be a key part of knowledge sharing, alerting, and system management. That changes at scale, but we adopted a pretty simple approach that has served Craftsy.com well.

AWS EC2 tags are a simple and powerful way to add all kinds of management layers and organization to your quickly growing cloud empire. In our world, we have 4 fundamental tags:

  • FQDN
  • enviro
  • role
  • app

A system might be tagged as prod,apache,craftsy to indicate it's a webserver in production for the craftsy application. All systems are uniquely tagged with FQDN (fully qualified domain name). This tag is what feeds our DNS naming system. Both the idea and the implementation are super simple... when an instance starts from any of AMI's, it executes a bash script from rc.local. That script looks up the FQDN tag for the instance, and sets a CNAME in route53 pointing to the ec2 public hostname. One of the nifty features of ec2 is you can call an instance by it's public hostname, and whether you are on the public internet, or internal to ec2, you'll get the proper resolution (public or private ip space).

I hope to commit an entire post to route53 in the future, but for now let it suffice to say you should abandon bind and use route53. Just. do. We'll talk more later.

Implementation of this setup is dependent on the Cirrus python package. Once you get that installed, and add your credentials you're ready to go. Probably a great idea to create an IAM user with only route53 add/modify permissions for this credential set. Cirrus gives you update_host.py:

[matthew@craftsy ~] ./update_host.py testing.craftsy.com ec2-xxx-yyy-zzz-ddd.compute-1.amazonaws.com -t 600

Assuming craftsy.com is a valid hosted zone in route53, you just set a cname pointing testing.craftsy.com to your ec2 hostname with a 5 minute TTL. Great! Now how do you get at that hostname?

A somehwat hidden feature in ec2 is the instance metadata. Basically anything you might want or need to manage or operate against is available in metadata. So, first we'll need our region, then we'll need to get the FQDN tag we established, and lastly we need our hostname:

export AWS_ACCESS_ID="BZZZZZZRRRTTT"
export AWS_SECRET_KEY="ohhaiohhaiohhai"
export EC2_AVAIL_ZONE=`curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone`
export EC2_REGION="`echo \"$EC2_AVAIL_ZONE\" | sed -e 's:\([0-9][0-9]*\)[a-z]*\$:\\1:'`"

Now let's get our FQDN tag:

iid=`curl -s http://169.254.169.254/latest/meta-data/instance-id`
FQDN=`ec2-describe-instances $iid --region=$EC2_REGION|grep FQDN|awk '{ print $5 }'`

last, the instance's public ec2 hostname:

ec2_hostname=`curl -s http://169.254.169.254/latest/meta-data/public-hostname`.

The rest writes itself:

update_host.py $FQDN -c $ec2_hostname -t 600

So, by establishing a CNAME that is meaningful to us we're able to use that name in parts of the app, monitoring, reporting, or administrative systems. Rolling this into any AMI you use means that any time the instance inits, whether with or without a new public hostname, you and your applications will always be able to get there. 

Comments (0)

The comments to this entry are closed.