Archive for March, 2009

Managing code releases

Recently I decided to streamline my code release process. I use subversion for my source control which means I push code live by running svn up on each of our production servers. I’m lazy, so I wanted an easier way to do this all at once. The end result is a simple shell script that lets me run svn update commands on multiple servers at once. It shows me the status of svn on each server and gives me chance to confirm that everything is okay before going ahead with the launch.

This example assumes you have two servers (app1 and app2) that are using public key authentication. Obviously, you’ll need to modify this script to work in your own environment. Make sure you replace “/var/www/” with your own document root and change to the IP address of each production server.


# connect to each server and echo their current status
echo "Connecting to app1...\n"
ssh 'cd /var/www/; svn status --show-updates; exit'
echo "\nConnecting to app2...\n"
ssh 'cd /var/www/; svn status --show-updates; exit'
# add additional servers here as needed
tput smso
# confirm the release before publishing
echo "\nDo you want to publish these changes to production? (y/n)\n"
tput rmso
read answer
if [ $answer == "y" ]; then
    # if "y", proceed with the release
    echo "\nPublishing to production..."
    echo "\nPublishing to app1..."
    ssh 'cd /var/www/; svn up; exit'
    echo "\nPublishing to app2..."
    ssh 'cd /var/www/; svn up; exit'
    # add additional servers here as needed
    echo "\nDone"
    # if "n", cancel the release.
    echo "\nCanceled"

Too many DNS lookups in an SPF record

I recently noticed I was having new email deliverability issues. It surprised me since things had been going well since switching to AuthSMTP for our outgoing mail. The first thing I checked was my SPF record. It looked like this:

v=spf1 a mx -all

At first glance everything seems okay. Basically it says to include all A records, MX records, and to include the SPF records provided by Google Apps, AuthSMTP and Salesforce. Since that covers every legitimate sender, I finish it off with the -all which indicates a hard fail. Ok, so the syntax is good. You can’t tell that anything is wrong without digging a little deeper. When you actually try to evaluate it you’ll get this error message:

Results – PermError SPF Permanent Error: Too many DNS lookups

After a little research I found out that you are only allowed 10 DNS lookups and fetching the TXT and SPF records count toward that total. That means after you add in the A and MX lookups, we’re at 7 before we even look inside the includes. Let’s pull up the SPF record for Google Apps:


That redirect counts as another DNS lookup. That puts me up to 8 DNS lookups. Thankfully the Salesforce SPF record is nice and clean:

v=spf1 ip4: ip4: ip4: ip4: ip4: mx ~all

That leaves AuthSMTP:

v=spf1 ~all

Ouch! That’s 4 more lookups and the worst part of it is that doesn’t even do anything!

The first thing I did was take out the MX lookup since it’s redundant. I also replaced with which is what it redirects to anyway. Technically, this isn’t a good idea since Google could change it on me — but remember I don’t have a lot of options here. I’m just happy to see my revised record pass the test:

v=spf1 a -all

I also sent an email to the AuthSMTP team. They responded within 30 minutes saying that they would remove the extra DNS record and look at how they can clean things up.

I learned something tonight. Remember to count the DNS lookups in your SPF record. It turns out they can add up faster than points on a teenagers drivers license. And if you’re using a lot of includes like I am, remember to do periodic checks to make sure nothing has changed.


  • I wrote about Sending email through Gmail over a year ago. While I absolutely don’t recommend you try this anymore, it has some useful information on SPF records and email deliverability in general.
  • Kitterman have a great tool to help validate your SPF records.
Nowadays I recommend everyone use SendGrid for sending email

Doing cool stuff with Flash

A few months ago I encouraged my friend Kevin Musselman to start blogging. He had been doing a lot of interesting stuff with Flash and I told him should be writing it down and sharing it with the community. He took my advice and has some interesting posts up including:

If that’s the sort of stuff that interests you, head over there and subscribe. I’m looking forward to seeing what else he’s got up his sleeve.