Filed under tech

nsd is bloody awesome

So it took a few months, but I finally got around to converting my DNS slave. It took about an hour to play around with NSD and figure out how to make it read from a bind master. And the actual cutover was less than five minutes.

Probably the part I like best is how clean the config file is. Take a look at this:

server:
        ip-address: 0.0.0.0
        do-ip4: yes
        do-ip6: no
        logfile: "/var/log/nsd.log"

remote-control:
        control-enable: yes

key:
        name: MYKEY
        algorithm: hmac-md5
        secret: "my+secret+in+base64="

zone:
        name: example.org
        zonefile: /var/db/nsd/example.org
        request-xfr: 1.2.3.4 MYKEY

It's insane how easy this was. Sometime soon I will update my master and then I'll be done with bind for good.

Tagged , , ,

switching from bind

In the What's New page for the upcoming FreeBSD 10, I see that the venerable (infamous?) bind is being replaced with LDNS and unbound, I found the related project NSD, also created by Stichting NLnet Labs.

This is good news. I've always been looking for a bind replacement because it is so large and unwieldy.

Late last year I started researching NSD to replace my bind server, and I've more or less decided I would switch over to it in 2014. But interestingly NSD doesn't have the thing I wanted the most: a clean zone file format. The config files are all sane now, but NSD more or less keeps BIND's zone file format. This page has some examples. Contrast this with DJB's tinydns which has an excellent zone file format, really short and succinct and hard to type in wrong.

Nonetheless, a couple of the primary ROOT servers are run using NSD, so that's very good to hear.

Will report back on my NSD adventures after I switch over.

Tagged , , , ,

How to Teach Yourself Unix Systems Administration

A while back, someone asked if anyone had successfully taught themselves Unix. (It was in the context of systems administration, so I'm assuming the question from a Unix sysadmin POV.)

An interesting question.

I was reflecting back on a few years of Unix experience and decided to put pen to paper (so to speak) and jot down what it would take to teach yourself Unix.

(NB: the question of whether or not I have been successful at it is hard to answer. I suppose you would have to ask my colleagues or supervisors that. But in any case, my observations follow.)

One of my University professors taught a Unix sysadmin course for grad students. I was an undergrad at the time, and did not take it, but followed it closely. The material didn't seem very hard, and I wish I had been able to take it. I don't think the fact that it was a grad-level course was what held me back at the time, but more likely a lack of funds. But I think more than coursework or access to books, the primary factor in being able to teach yourself Unix is a hunger to do so. If you are not interested in the subject, then no amount of classes you take will help you. If you have a sincere desire to learn the subject, then lack of materials will not hinder you.

Almost all my computer-related college courses had to do with Unix in some shape or form. If you paid proper attention, they Unix part of it was soaked into your bones via osmosis.

But that was many years ago.

There are a lot more formal courses now on Unix and Unix systems administration nowadays. Certainly there are more books and online materials. So the young person who is looking to enter this field certainly has more resources to access that they can take advantage of.

Get a Unix Job

If you are in college or about to get into college, definitely try and hit up a college Unix lab. They are always looking for cheap labor and you will learn a LOT. Also, they don't require a glamorous resume. If you know a bit and are hungry to learn, you can probably snag the job.

Get Unix books

If you are serious and have some money to spend, go and get the Evi Nemeth (Prentice Hall) and Aileen Frisch (O'Reilly) Unix Sysadmin books. They are both excellent. The one by Tom Limoncelli is pretty good too.

These are all sysadmin books by the way. There are books that have solely to do with Unix that you should also get.

Run Unix

I think it goes without saying that you should already be running a Linux or a BSD on your home PC already. Don't dual boot! That only will tempt you to boot back into Windows when you find you can't do something in Unix. Kill Windows and jump into Unix cold turkey. Start off with something easy like Ubuntu or Fedora, and then work your way to something more powerful like Debian or FreeBSD (which have steeper learning curves).

Google It

You have something now that I did not have when I started out -- Google. Use that to your advantage. If you are not sure of something, Google it, odds are there have been tons of people who have gone before you and have had the same doubts.

Join a User Group

Definitely join your local LUG or BSD user group. Subscribe to their mailing lists and stay on it.

Learn a Programming Language

In addition to learning Unix, if you don't how already, you should be teaching yourself how to program. Not just shell scripting, although that is a start. Pick one of Python, Ruby, or Perl, and learn it inside and out. A good Unix person is a good programmer. Unix is so powerful because it gives you a rich suite of tools to program and is probably the most programmer-friendly of all the modern operating systems. Even if you never use Unix in the future, you will find that the ability to program your way out of certain problems to be invaluable. As a sysadmin, not a single day goes by where I don't write or use some dozen line throwaway script that just makes my life easier. You won't believe me now, just do it, and trust me on that.

Join a Professional Organization

There will be things that you just won't be able to teach yourself, either because it (1) takes experience or (2) is very expensive. #1 will come with time and with jobs of greater complexity and responsibilities. It won't hurt to to join LOPSA or Usenix SAGE later on, they have a lot of people talking about best practices and such which is very useful, even if you don't plan on becoming a sysadmin. As for #2, stuff like SANs and NASs and multipath and fiber channel are harder to learn on your own with homebrew equipment unless you've got a job where you can play around with stuff like that, but it will come as you graduate to better jobs.

Don't Quit

It will all be a bit obtuse and difficult at first. Unix has a fairly steep learning curve. Stick with it. One day, probably a year or so into it, things will just "click". You will see the beauty of the command line tools and how awesome it is to glue them together with pipes. You will behold the elegance of putting everything in text files and the value of a really powerful text editor and shell. Later on, you will invariably find yourself on a Windows or Mac box and discover that things that are easy and powerful on that Unix box of yours is either impossible or very hard to do on non-Unices -- this is less true with Mac nowadays since it is Unix under the hood, but you get the gist. Once you reach that point, you will have "gotten" Unix and will begin to love it.

Learn to Love It

I think it was Paul Graham who talked about this in one of his essays: during the dot com boom someone came to him and told them they were going to start a company and that it was going to be based on Windows. The person assured him that the company was going to be a success because they hired Windows "experts". Graham thought to himself that the company was doomed to failure. The reason being that in order to be an expert at something, you had to have used it for several hundreds of hours (the equivalent of 10-man years); not just at work or school, but in your own free time. In order to become an expert at an OS, you needed to love it and use it all the time. Graham knew people loved Unix and used it all the time. On the other hand, he knew that no one had any real love for Windows, so these so-called experts probably weren't. Sure enough, the company went under a short time later.

Tagged

stop your ISP from hijacking your DNS

If you're like me, you use some local ISP for your primary, non-work, Internet connection. Since I'm in NYC, I use Verizon's FIOS service. Although it is pricey, the speeds I get are decent (by American standards).

However, Verizon does one thing that they do that I absolutely hate: THEY HIJACK YOUR DNS.

What is DNS hijacking? Wikipedia has a fairly good entry on the subject.

Basically, if you make an invalid DNS request, instead of returning an error (NXDOMAIN) to your browser as required by the Internet standards, they instead route the page to one of their own pages, usually to display advertising. This breaks all sorts of things, these are described in gory detail in the article above—but such a blatant violation of an Internet RFCs is enough to justify an outcry.

Presuming you use some sort of Unix box, there's a couple of solutions for this:

  • You can change the permissions flags on /etc/resolv.conf to be immutable (chflags on FreeBSD, chattr on Linux), although I consider this to be a bit heavy handed.
  • You can tell your DHCP client to not update resolv.conf (via dhclient.conf)
  • But I think the easiest is way to solve this problem is to override the settings on your wireless router to send hardcoded DNS settings to DHCP clients instead of accepting the ones sent by the ISP. The added advantage here is that it works for all operating systems and clients in your subnet.

Which DNS servers to use instead? Google offers a service called Public DNS that won't hijack your DNS:

  • 8.8.8.8
  • 8.8.4.4

Although you should keep in mind that Google can and will use your name lookup data for advertising purposes.

Lastly, this article assumes this is for a home or personal network. If you are running a business or other organization, you really should be using your own DNS servers.

Tagged , ,

du df size discrepancy

Sometimes people report a huge size discrepancy between the output of du and df. There are two main reasons for this:

  • open file descriptors
  • mounted filesystems hiding unmounted filesystems

With regards to open file descriptors, check that any processes that are running are not holding open files. Shut down Apache, or Oracle, or MySQL, or Postgres, etc. That should usually do it.

The other thing you can check is to see that mounted filesystems are not masking (or hiding unmounted filesystems). Unmount all filesystems and check again, you will see what has been hiding behind the mounted FS.

Tagged