Dholdrums

It's 2018. There's pretty much no excuse for not having two factor authentication (2FA) setup on on all your accounts.

One of things you should have 2FA setup on is your servers that you SSH into. Of course, you should be disabling SSH password logins, and only use SSH keys. But you may have some bastion hosts that you allow password logins on, and on those server, you should setup TOTP authentication.

Here's how to do it on FreeBSD:

• Install Google Authenticator (or some other TOTP-based 2FA program) on your phone or device.
• Install the Google Authenticator PAM module:

$ sudo pkg install pam_google_authenticator

• Next, generate a token for your server and answer some simple questions:

$ google-authenticator

Do you want authentication tokens to be time-based (y/n) y
https://www.google.com/chart?chs=200x200&chld=M|0&cht=qr&chl=otpauth....
Your new secret key is: SN6DNZ2W7Z2R56BL
Your verification code is 934157
Your emergency scratch codes are:
  38875904
  94027394
  76418491
  71483023
  75284805

Do you want me to update your "/home/user/.google_authenticator" file (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds and in order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with poor
time synchronization, you can increase the window from its default
size of 1:30min to about 4min. Do you want to do so (y/n) n

If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

• Go to the URL given, and you will see a QR code similar to the follow. Add it into your Google Authenticator app.

• Add the following line to /etc/pam.d/sshd:

auth  required  pam_google_authenticator.so

• Restart SSH: sudo service sshd restart

You should know be prompt for your TOTP token when you log in now:

$ ssh example.org
Password for user@example.org:
Verification code:

I started using Let's Encrypt recently. It is as good as they say it is.

Let's Encrypt allows you to get free SSL certs for your site, provided that you can prove that you own the domain. They do this by putting a nonce into your webroot, which they then pull via http. The whole process took all of 30 seconds, and that includes the install of the actual software.

$ sudo pkg install py27-letsencrypt-0.1.0

$ sudo letsencrypt --webroot -d example.org --webroot-path \
        /usr/local/www/data/example.org certonly

I was blown away by how easy it was.

Did I mention the certs are free?

The certs have an expiration date of 3 months, which is fine since they are free (free!) and can be easily renewed, I've heard reports of people re-generating them from scripts in cron.

This is great news. This basically means we can slap SSL certs on pretty much everything, and that's a good thing. Like this blog for example. Hmm...be right back.

I switched jobs a few months ago, so I had to return my work laptop. It was an HP EliteBook 2560p.

The thing ran BSD very poorly due to bad/nonexistent support for HP's proprietary hardware. I probably would have kept at trying to get FreeBSD to run properly on it, but my work used Google Hangouts for pretty much everything, so I was relegated to using Ubuntu on it.

I'm not a terribly big fan of Ubuntu, but all I really need is a proper terminal (I use gnome-terminal with Inconsolata fonts), a good shell, and Firefox, and Ubuntu does the trick with in that department, so there you go.

My new job gave me a brand new shiny laptop. A Macbook Pro Retina 15. It's supposed to be the shiznit, according to all reports.

But I hate it it. It's Unix, but barely. I definitely dislike it more than Ubuntu. It's a huge monstrosity. I tried using MacPorts to make my environment BSD-like, but nothing worked. Part of the problem is that the new job doesn't allow us to keep a running workstation on our desk, so I don't really have a server at work to connect to, so it's very annoying. It's a powerful machine to be sure, but for the sort of work I do, I barely use 1% of its power.

Right now, I'm sort of managing by running a FreeBSD vmware instance on the laptop. Whenever the laptop is running, that is my "server".

But I've given work trying to get things right and resigned myself to this ghetto.

But at home, I refuse to use this MacBook for any of my work. So I went to Amazon and bought a used HP EliteBook 2560p. In retrospect, I probably should have looked at similar sized Thinkpads, but it was sort of an impulse buy. I got it for only $180.

My wife was horrified, but it works great for what I need it to do.

In case anyone decides to run BSD on it and wants to make it work, I sort of made some progress:

• In the BIOS, switch the hard drive to not use AHCI, use IDE instead
• in the installer, don't use GPT, use BSD labels instead
• to get X to run properly, put kern.vty=vt in loader.conf

I had to go on a long trip, so I reverted to using Ubuntu on it just have something useful running, but I will try to get it to work with BSD again soon.

I was once given a netbook, ca. 2009, in the golden age of netbooks.

It was a Dell mini 10v. My boss at the time wanted to upgrade to a larger screen laptop, so I got the netbook.

I thought I would hate it, the keyboard was much smaller than I was used to, and I initially hated the 10in screen.

I used it for 3 years and ended up loving it. Reasons:

• the fact that it was so small and light. It was a pleasure to carry it around in my messenger bag, it weight about 1300grams (2.9lb)
• I got used to the keyboard fairly quickly
• it ran Ubuntu (and later Ubuntu netbook remix); while Ubuntu isn't BSD, I prefer it over Windows/Mac OS
• best of all: it had Dell's equivalent of a retina display, the screen was ultra sharp and colorful

I never noticed it being slow. In fact, the only downside to it that I remember was that the screen (10.1 inches) was sometimes too small to view a page in Firefox. But I only noticed that sometimes.

It was work hardware, so I didn't pay for it myself, but the pricepoint ($300) was great as well.

But around 2010, the netbook market died. Mostly for two reason:

• newer versions of Windows couldn't handle the slower Intel atom processors and smaller screens of netbooks
• tablets came in and mostly supplanted the category

Dell discontinued it's mini line, and so did everyone else. The last time Asus made an Eee PC was in 2013, and it was dead for a few years before that.

A shame, a netbook would be perfect for me right now. I mostly just live in a terminal with Firefox running. That's about it.

But alas, everyone stopped making them. The closest thing I can find are 12.5" laptops. They are okay, but netbooks would be better.

Oh well.

So Yahoo's silly DMARC policy, which they introduced in April of 2014, bit me today.

I setup a mailing list for someone for a group they volunteered with, sent out a couple of test messages to ensure everything was working, and handed it over.

Everything was working until they mentioned that they didn't get a message from the main organizer of the group. The only reason they knew it was even sent was because I was subscribed to the list myself, and mentioned to them in passing "Oh, so-and-so sent an email" when I first saw the message come through on my phone.

A few hours later, when they were sitting at their computer, they mentioned that they hadn't received the aforementioned email from the organizer. A little digging revealed the mail was in the GMail spam folder. A little further digging revealed that the original poster was using a Yahoo email address.

I'm not going to go into why I think Yahoo's policy is dumb. They have a right to publish their DMARC policies as they see fit. It does however break electronic mailing lists, and apparently Yahoo doesn't care too much about that.

So the rest of this post is about how we dealt with this problem.

My first inclination was to ask the user to use an alternate (GMail) address instead of their Yahoo one. But I rejected this because this was a nontechnical user on a nontechnical list and I didn't want to inconvenience them for their ESP's stupidity.

My next option was to turn on list munging for my list. This would rewrite the From: line of all the messages sent to the list.

So something like this:

From: Joe Blow <jblow@yahoo.com>
To: Foo List <foo@example.org>

would turn into something like this:

From: Joe Blow via foo <foo@example.org>
To: Foo List <foo@example.org>

I knew this was possible with GNU Mailman, and the setting itself is pretty simple:

from_is_list = 1

But the problem is that that this munges everyone's address, not just those from yahoo.com. It was then I discovered these two gems:

dmarc_quarantine_moderation_action = 1
dmarc_moderation_action = 1

Basically this means that if dmarc_quarantine_moderation_action is turned on, then dmarc_moderation_action is taken on sites that basically have p=reject. In this case, the 1 means to munge. So basically, munging is done only on sites with this DMARC policy.

These features are available in GNU Mailman version 2.18 and greater.

Major thanks to the GNU Mailman devs for implementing this—they did it pretty quickly after the start of this fiasco too, judging by the release dates.

Even before I stumbled onto the jewels that are Pelican and other static site generators, I had longed to replace my gallery albums.

Gallery was one of the first good open source image gallery systems out there. It did an okay job, but like WordPress, it was written in PHP and massively overengineered and overfeatured.

I once spent months setting up Gallery for a client. Apart from building and installing the software (pretty easy with FreeBSD's ports) and actually getting the images together, most of the time spent was trying to integrate the gallery output pages into the look and feel of the client's existing website. That was massively painful, but in the end, it worked beautifully.

Some time later, some devs overhauled the website, and my work was put on the shelf. A bit after that, the powers that be decided the new website was aesthetically challenged (a euphemism for ugly), and wanted me to revert back to my old site. Which I did. Except that Gallery broke. I'm not sure how. The database and files were restored to their exact states (we had been meticulous about backups), but we could never get it to look the same again. I talked to some gallery devs, and everyone just recommended I upgrade from gallery 2 to gallery 3.

In the end, the gallery images were only a small subsection of their website, so we ended up not restoring it, deciding it was not worth the trouble.

Since then, I've longed for something that would just spit out raw HTML and CSS, just like pelican.

It looks like Sigal is it.

It's not perfect, by default it wants to spit out new-agey Javascript galleries, blech. But it works with Jinja2, so it should be pretty easy to make it do what I want.

Stay tuned!

I recently decided to publicly mention a small project I've been hacking away on.

It was more or less in the I'm just hacking on some code state for a while. And then in the past year, I cleaned it up a bit, added an interface, and made it (barely) functional -- if a bit feature-incomplete.

So I wanted to show it to other people, and what better way to do that than at a lightning talk at a conference?

But I wanted to have more than a GitHub page to point to, and I really didn't want to spend the time to author up a fancy website (although I can understand some HTML/CSS, I'm not a web designer by any means), so I decided to give Pelican a shot at this job.

Pelican, although mostly designed for blogs, also does a decent job at static web pages.

I got it up and running in about an hour or so. I spent most of that time finding a good theme which I liked.

I'm very happy with the results: recalc.org.

So my overall consensus is that if you need a fairly simple website, and like the [edit with text editor] and then [run make to see your output] way Pelican works, it's the tool for you.

So a couple of years ago, I got tired of all the brute force ssh attacks on my servers.

I wasn't particularly worried about anybody getting in: on most of my boxes, there are only one or two allowed logins, and I restrict who can log in using the AllowUser parameter in /etc/ssh/sshd_config. There are also other safeguards in place.

Still, if someone has unsuccessfully tried 30 times to log in as "root" on my box, I'd like to avoid having them come into contact with any of my hosts, so enter SSHGuard.

SSHGuard, along with its more well known brethren, DenyHosts and Fail2ban, all work roughly the same way. They monitor your logs, and if they see some predetermined number of failed logins, they block the ip.

The devil is in the details. They all vary on what kind of logs they can parse. They all can parse ssh logs, but some can also handle Apache logs, mail logs, ftp logs etc. They all seem to be able to talk to iptables directly, but others can also understand pf, ipfw, etc.

For a couple of years, I used DenyHosts on my FreeBSD box. It was a pain to setup, and as I found a few months ago, didn't work properly. DenyHosts parsed auth.log well enough, the problem was that it didn't speak pf. So in order to get it to work with pf, I had to write wrapper scripts. The interface was easy enough, you give a path to a script to add an IP and another path to remove an IP. The scripts (de)populated some file and would call pf to reload the table. The following would be the relevant pf rules:

table <denyhosts> persist file "/var/db/denyhosts/pftable"

block in log quick on $EXT_NIC from <denyhosts>

This all would have been well and good until I recently noticed that my blacklist file was growing at an ever steady rate, and included duplicates. Basically, DenyHosts kept adding the same IPs, and never took any out. There were easily thousands of entries.

For a while I thought it was a problem with my scripts, and I debugged the hell out of it (not easy when the only way to test it is to wait for someone to brute force your ssh server) and finally realized that it was some bug in DenyHosts. I probably should have filed a bug, or brought it to the attention of the developers, but by that time, I was pretty disgusted with everything, so I just turned it off altogether

A few months ago, someone was asking on the FreeBSD mailing list about how to secure their server and someone suggested SSHGuard. I don't know how I didn't find out about this project earlier. It's been around since 2007 at least.

SSHGuard has the following features that I liked:

• parses many types logs, not just ssh
• can talk to pf directly
• has a port (security/sshguard-pf) which works out of the box with little to no configuration
• written in C, so no dependencies on scripts demanding the interpreter (other programs seem to eat a lot of resources)
• didn't I mention that the needed configuration is tiny?

Here's what I did to get it working:

• installed the port
• added the following pf rules:

table <sshguard> persist

block in log quick on $EXT_NIC proto tcp from <sshguard> \
to any port 22 label "ssh bruteforce"

• added the following to /etc/rc.conf

sshguard_enable="yes"
sshguard_watch_logs="/var/log/auth.log"

• created the file /usr/local/etc/sshguard.whitelist and put in my whitelist entries (SSHGuard is really flexible about the format of this file)

And that was pretty much it.

My one minor gripe is that SSHGuard's blacklist DB file is not easily parsable, so I'm stuck to dumping the pf table (pfctl -t sshguard -T show) to look at blacklisted entries. Not a big deal.

And that was it. SSHGuard has worked perfectly for me. I will soon change the pf rule to block blacklisted IPs access to everything on my hosts, not just port 22.

Finally done!

This blog is now statically generated by Pelican!

It was incredibly easy to move over. The WordPress XML import more or less worked without a hitch, and I'm very satisfied with the end result.

I will craft my own theme one of these days, but until then, one of the precooked ones is fine.

In the 3 years or so that this blog has been up, I've always been uneasy about it. A couple of reasons for this:

Firstly, Wordpress, the underlying platform, consists of a bunch of PHP scripts wrapped around a MySQL database. By far the most frequent security issues I've seen come sites running PHP. Drupal, Joomla, et al. are all easily hackable and WordPress is no exception. I update WordPress religiously on my system and I'm still afraid that someone is going to use some zero-day PHP exploit to hack me.

Less dangerously, but more annoyingly, editing articles for the blog requires that I log into my site local instance of WordPress and use the app to edit the article. This is not terribly hard, but just hard enough that I don't like doing it. Why can't I just fire up vi (startup time = ~300ms) and start writing? Hence in 3 years, I've written maybe 10 articles total. In fact, I've just played with Pelican for a bit and it was incredibly easy to add and edit articles. I'm now writing this post in WordPress and by comparison it is slow as molasses, and inelegant to type this into a web window.

I started looking at static site generators a while back. The first was Blosxom, which came out 12 years ago apparently. I first investigated it in 2004. It had promise back then, but was still rough around the edges.

I have since discovered Pelican, thanks to dotCommie. It's written in Python, uses Markdown/reStructured Text as input, and with the magic of templates (it uses Jinja), generates totally static content. Which is all I need. I've been playing around with it and like what I see so far. I will switch the entire site over when I find some free time.