Cloaked Master, BIND, DNS Made Easy and Anycast slaves.

I have happy news today. DNS Made Easy has announced that it's expanding it's Secondary DNS system to run on their extensive anycast network.

 Anycast? Huh? Ok... For those of you who don't stay up at night reading IETF documents -- anycast is, in basic terms, the ability to attach multiple computers (in different locations) to a single IP. It's a lot more complicated how it works with BGP announcements and AS numbers, but the end of the day, a user querying your DNS service should get the closest server to them. You might publish 3 ip's but have 20 geographically diverse systems ready to respond.

The exact locations of the DNS Made Easy anycast servers don't appear to be published, but http://www.dnsmadeeasy.com/s0306/aboutus/net.html gives a little information about where some of their servers are. (Oddly, there doesnt seem to be any in Canada [psst, i suggest Vancouver (Harbour Center) and Toronto (151 Front Street)]) ... anyway. So, to take advantage of this awesome level of redundancy would normally cost you just buckets of money. Running an IP Anycast network is sadly way too technical for the average company, and I can't think of a better service (except maybe email) to outsource to the experts.

If you've got loads of cash to throw at it... you could go with someone like Dyn. But I see no indication that their service is any better than that of the MUCH cheaper DNS Made Easy ... Anyway...

I, like most web nerds, am not willing to manage my DNS from a  web control pannel. Too much hassle, hard to back up, imprecise -- and I'm constantly losing the mkpasswd password needed to keep these things secure online. So that's out. I like bind. I've been working with bind for many years -- and even worked with another c-dev on some serious improvements to db-driven bind environments with bind-dlz. I get bind. It works. But, then I don't want to run my own anycast -- and there's always the issue with your own DNS servers getting flooded off by some script kiddie. So how do you get the best of both worlds?

You run bind on a secret server somewhere, you publish the slaves as your domain DNS records and cloak the master (sometimes called a hidden master)... 

 HowTo: (more after the break)

1. Sign up for an account with any anycast provider that offers SECONDARY DNS services. (I suggest this affiliate link that might give me a credit *cough* DNSMadeEasy.com)

2. Install Bind... These instructions are for ubuntu.

 apt-get install bind9 

3. The configs

/etc/bind/named.conf.options

options {
directory "/var/cache/bind";
auth-nxdomain no; # conform to RFC1035
listen-on { 192.168.1.100; }; # your ip address. I've converted mine to a lan ip here, but you should use your public ip
listen-on-v6 { none; }; # if you have an ip6 addr, configure it here.
allow-transfer { 63.219.151.12/32; 64.246.42.203/32; 208.94.147.135/32; 205.234.170.139/32; };
also-notify { 63.219.151.12; 64.246.42.203; 208.94.147.135; 205.234.170.139; };
allow-update { none; } # we're not a slave so no updates
query-source address 192.168.1.100; #your ip address.
notify-source 192.168.1.100 port 53; # your ip address.
notify Yes; # this is required for operation when the master is hidden as the remote servers cant rely on the SOA
recursion no; # we dont want to be a resolver.
};

logging {
channel simple_log {
file "/var/log/named/bind.log" versions 3 size 5m;
severity debug;
print-time yes;
print-severity yes;
print-category yes;
};

channel audit_log {
file "/var/log/named/audit.log" versions 3 size 5m;
severity debug;
print-time yes;
print-severity yes;
print-category yes;
};

channel query_log {
file "/var/log/named/query.log" versions 3 size 5m;
severity debug;
print-time yes;
print-severity yes;
print-category yes;
};

category default { simple_log; };
category general { simple_log; };
category security { audit_log; simple_log; };
category config { simple_log; };
category resolver { audit_log; };
category xfer-in { audit_log; };
category xfer-out { audit_log; };
category notify { audit_log; };
category client { audit_log; };
category network { audit_log; };
category update { audit_log; };
category lame-servers { audit_log; };
category queries { query_log; };

};

This will result in pretty much everything you do on your bind server being logged. I've had some issues in the past with ubuntu's security preventing writing to /var/log/named so reload your server and check that the log files are being created. The IP addresses listed here are for DNS Made Easy's new ip anycast service -- yours will probably be the same, but check it.   You can comment out the also-notify addresses until you have the rest of your zones setup, working and added to the dns made easy web control panel. By adding these allow-transfer and also-notify entries, your server will mostly ignore the SOA record's directives. This will also have the side effect of creating a notify to update master, which means your changes will be almost instantly published to your slaves. Because of this, we'll set the SOA record's timeout pretty high at 1 day, as this will let your master go down for a while, without sacrificing update speed.

 /etc/bind/named.conf.local

zone "example.org" {
        type master;
        file "/etc/bind/domains/example.org";
};

You'll add as many of these records as you have domains. If you're using a DB-Driven backend like Bind-DLZ... you'd add the zones as normal here.

/etc/bind/domains/example.org

$ORIGIN .
$TTL 180
example.org. IN SOA ns1.example.org. kevin.example.org. (
2010010401 ; serial
1d ; refresh
5m ; retry
1w ; expire
10m ; minimum
)

NS ns1.example.org.
NS ns2.example.org.
NS ns3.example.org.

A 192.168.1.101 ; just another host any ip is fine.

$ORIGIN example.org.
www A 192.168.1.101
ns1 A 208.94.148.13
ns2 A 208.80.124.13
ns3 A 208.80.126.13
;other hosts as desired

The ttl is set to 3 mins, the refresh to 1day (cuz this is an update-based server) and the expire is set REAALLLY long in case your server dies and the secondary ns's need to keep running without checking in.The serial MUST BE INCREMENTED EVERY TIME YOU CHANGE THE ZONE FILE. If you dont, the slaves wont pick up the changes until the expiry *very bad*. I use the format YYYYMMDDRR where RR = revision. Which caps you to 99 changes per day -- which works for me. The kevin.example.org is an email address that equates to kevin@example.org. Don't forget to add a . at the end of domains.

So... once that's all setup. Restart bind and test your domain

/etc/init.d/bind9 restart 
dig @192.168.1.100 example.org

The response should list your .101 ip's, the ns records etc... ok, so now go over to your dns provider and add the domain to the secondary service... it'll ask you the domain name, and the ip address of your master server (192.168.1.100 in this example, but it must be a public ip)

Get a cup of coffee... it takes a few mins for it to be added to the system.

Ok, so its added... next, go and uncomment those allow-transfer, also-notify option lines and reload.

Then go into your DNS Registrar (eg where you purchased your domain name from) and update your nameservers. List

ns1.example.org               208.94.148.13
ns2.example.org               208.80.124.13
ns3.example.org               208.80.126.13

As your nameservers. NOTE: We don't include the master nameserver ip here, nor the notify/transfer servers. These are the anycast IP's. Only the DNS provider knows where your master is, and its not part of your zone file, nor listed on your domain. It will ONLY be used by the dns provider by ip, and never by your clients. You might be thinking, but what would it hurt to add it here... well, it will mean that 1 of four randomly selected addresses for resolving your domain is NOT anycasted, and it will slow the process down 1 in 4 times. At least, in theory. In reality, many clients will try all the ips and use the first response. Anyway, it wont really help. What it will do however, is point out to anyone looking to flood your DNS where your master is located.  The DNS providers servers will be stronger, and more well monitored than yours -- though, I've heard that they might remove your domain from their system if you're attacked [which I think sucks, but may be standard industry practice], anyway, it'll be better than your little dedicated colo in any event. So list only the needed IPs, in this case, less is more.

Finally do a 

dig example.org +trace

Wait until your new IP's show up in the 2nd to last step.

Now. Update the serial to a new number (rev 01 to rev 02)... and

/etc/init.d/bind9 reload

You should see 

04-Jan-2010 16:09:12.199 xfer-out: info: client 205.234.170.139#45106: transfer of 'example.org/IN': AXFR-style IXFR started
04-Jan-2010 16:09:12.199 xfer-out: info: client 205.234.170.139#45106: transfer of 'example.org/IN': AXFR-style IXFR ended
04-Jan-2010 16:09:12.203 xfer-out: info: client 63.219.151.12#39959: transfer of 'example.org/IN': AXFR-style IXFR started
04-Jan-2010 16:09:12.203 xfer-out: info: client 63.219.151.12#39959: transfer of 'example.org/IN': AXFR-style IXFR ended
04-Jan-2010 16:09:12.227 xfer-out: info: client 208.94.147.135#55417: transfer of 'example.org/IN': AXFR-style IXFR started
04-Jan-2010 16:09:12.227 xfer-out: info: client 208.94.147.135#55417: transfer of 'example.org/IN': AXFR-style IXFR ended
04-Jan-2010 16:09:12.231 xfer-out: info: client 64.246.42.203#39776: transfer of 'example.org/IN': AXFR-style IXFR started
04-Jan-2010 16:09:12.231 xfer-out: info: client 64.246.42.203#39776: transfer of 'example.org/IN': AXFR-style IXFR ended

In /var/log/named/audit.log As all of your notified nameserver's grab a copy of the new zone.

Finally, test your ns's one-by-one with a 

dig @ns1.example.org -t soa example.org 
dig @ns2.example.org -t soa example.org
dig @ns3.example.org -t soa example.org

And make sure that you see your NEW serial.

If you don't.... theres a good chance theres an iptables firewall in the way. You might be thinking Nah, not possible, i opened port 53 udp to the world.... well, actually... see... zone transfers and other secure dns operations actually happen over TCP... its just too easy to spoof a udp connection. So, open up the tcp port 53 as well. It doesn't need to be accessible to the public though, just to the 4 servers listed above. 

Upon opening the firewall, you should start to see the axfr's start... if you don't you can force a notify by bumping the serial number up one and reloading bind. You should see the AXFR now.

And thats how you setup an IP anycast, stealthed master, dns service. Easy eh?