Posts tagged ‘tutorial’

How to use variable variables in PHP

One of the biggest time-savers in PHP is the ability to use variable variables.  While often intimidating for newcomers to PHP, variable variables are extremely powerful once you get the hang of them.

Variable variables are just variables whose names can be programatically set and accessed.  For example, the code below creates a variable called $hello and outputs the string “world”.  The double dollar sign declares that the value of $a should be used as the name of newly defined variable.

$a = 'hello';
$$a = 'world'
echo $hello;

When I started with PHP about 10 years ago, everyone was still using global variables.  That meant that anything you passed as a GET variable could be used as a local variable.  It was very convenient, but unfortunately not very secure.  For me, typing $HTTP_GET_VARS[‘count’] just wasn’t as fun as being able to use $count.  I found myself adding long declaration lists to the top of my files that did nothing but convert my GET/POST variables to local variables.  My code started to look like this:

$salutation = $HTTP_GET_VARS['salutation'];
$fname = $HTTP_GET_VARS['fname'];
$lname = $HTTP_GET_VARS['lname'];
$email = $HTTP_GET_VARS['email'];

Do that for a couple dozen variables and you’ll start telling yourself there has to be a better way.  Nowadays you can use $_GET instead of $HTTP_GET_VARS, but the better solution is to use variable variables. Now my code looks more like this:

// create an array of all the GET/POST variables you want to use
$fields = array('salutation','fname','lname','email','company','job_title','addr1','addr2','city','state',

// convert each REQUEST variable (GET, POST or COOKIE) to a local variable
foreach($fields as $field)
    ${$field} = sanitize($_REQUEST[$field]);

This has several benefits.  I reduced 14 lines of code down to 3.  I now have one place to sanitize all my external input. And if I ever decide to change a variable name, I have one less place in my code to fix.

This benefit of this technique increases as you use the $fields array throughout your code.  I now utilize the $fields array when saving my form data to the database.  I use it for loading existing user values from the database.  I use it for passing my form fields back to smarty:

$form = array();
foreach($fields as $field)
    $form[] = $_REQUEST[$field];

Variable variables have become one of my favorite features of PHP. They’ve allowed me to tighten up a lot of my code and made it a lot more maintainable.

Have you done anything cool with variable variables?  What other PHP tricks have revolutionized the way you write code?


How to use curl_multi() without blocking

You can find the latest version of this library on Github.

A more efficient implementation of curl_multi()
curl_multi is a great way to process multiple HTTP requests in parallel in PHP. curl_multi is particularly handy when working with large data sets (like fetching thousands of RSS feeds at one time). Unfortunately there is very little documentation on the best way to implement curl_multi. As a result, most of the examples around the web are either inefficient or fail entirely when asked to handle more than a few hundred requests.

The problem is that most implementations of curl_multi wait for each set of requests to complete before processing them. If there are too many requests to process at once, they usually get broken into groups that are then processed one at a time. The problem with this is that each group has to wait for the slowest request to download. In a group of 100 requests, all it takes is one slow one to delay the processing of 99 others. The larger the number of requests you are dealing with, the more noticeable this latency becomes.

The solution is to process each request as soon as it completes. This eliminates the wasted CPU cycles from busy waiting. I also created a queue of cURL requests to allow for maximum throughput. Each time a request is completed, I add a new one from the queue. By dynamically adding and removing links, we keep a constant number of links downloading at all times. This gives us a way to throttle the amount of simultaneous requests we are sending. The result is a faster and more efficient way of processing large quantities of cURL requests in parallel.

function rolling_curl($urls, $callback, $custom_options = null) {

    // make sure the rolling window isn't greater than the # of urls
    $rolling_window = 5;
    $rolling_window = (sizeof($urls) &lt; $rolling_window) ? sizeof($urls) : $rolling_window;

    $master = curl_multi_init();
    $curl_arr = array();

    // add additional curl options here
    $std_options = array(CURLOPT_RETURNTRANSFER =&gt; true,
    $options = ($custom_options) ? ($std_options + $custom_options) : $std_options;

    // start the first batch of requests
    for ($i = 0; $i &lt; $rolling_window; $i++) {
        $ch = curl_init();
        $options[CURLOPT_URL] = $urls[$i];
        curl_multi_add_handle($master, $ch);

    do {
        while(($execrun = curl_multi_exec($master, $running)) == CURLM_CALL_MULTI_PERFORM);
        if($execrun != CURLM_OK)
        // a request was just completed -- find out which one
        while($done = curl_multi_info_read($master)) {
            $info = curl_getinfo($done['handle']);
            if ($info['http_code'] == 200)  {
                $output = curl_multi_getcontent($done['handle']);

                // request successful.  process output using the callback function.

                // start a new request (it's important to do this before removing the old one)
                $ch = curl_init();
                $options[CURLOPT_URL] = $urls[$i++];  // increment i
                curl_multi_add_handle($master, $ch);

                // remove the curl handle that just completed
                curl_multi_remove_handle($master, $done['handle']);
            } else {
                // request failed.  add error handling.
    } while ($running);
    return true;

Note: I set my max number of parallel requests ($rolling_window) to 100 5. Be sure to update this value according to the bandwidth available on your server / servers you are curling. Be nice and read this first.

Updated 3/6/09: Fixed a missing semi-colon. Thanks to Steve Gricci for catching the typo.

Updated 4/2/09: Made some changes to increase reusability. rolling_curl now expects a $callback parameter for a function that will process each response. It also accepts an array called $options that let’s you add custom curl options such as authentication, custom headers, etc

Updated 4/8/09: Fixed a new bug that was introduced with the last update. Thanks to Damian Clement for alerting me to the problem.


How to start MAMP on port 80 without a password

I’m a big fan of MAMP. It’s the fastest way for anyone to get set up with a local PHP/MySQL development environment on a mac. One of the small annoyances with MAMP is that it requires you to enter your password all the time if you want to run it on port 80 (which I do). To be fair, it’s got more to do with UNIX security than MAMP… but it’s still bloody annoying!

I tried Steve Stringer’s technique of using launch daemons, but it just couldn’t get it to work for me.

The trick to getting MAMP to start behind the scenes is knowing that all that pretty GUI does is call a couple shell scripts. Specifically, those scripts are /Applications/MAMP/bin/ and /Applications/MAMP/bin/ (assuming you installed MAMP at the default location).

The second thing you should know is that must be run as root, but must be run as the current user. I created a new shell script to call those scripts appropriately:

sudo /Applications/MAMP/bin/
exit 0

I then added added an exception for that script to my sudoers file so I didn’t need to enter a password when I used sudo. The easiest way to add this exception is to use the ‘visudo’ command as root.

Finally, I used Automator to wrap the whole thing up as an application I could add to my dock. It works! One less daily annoyance in my life!

Since writing this, Damian Gaweda has posted a more elegant solution that’s worth checking out.

How to make OpenDNS shortcuts work in Safari

As I’ve written here before, I am a huge fan of OpenDNS shortcuts. I’ve always hated switching over to Safari for the simple reason that my shortcuts didn’t work. You might laugh, but it’s hard to go back to typing full urls once you’ve gotten used to single letter aliases. Before you call me lazy, try it for yourself. I promise, you’ll never go back.

Anyway, last night I finally found the secret to making shortcuts work in Safari.

Open up your System Preferences. Go to the Network pane. Select the TCP/IP tab and enter “” or “” into the Search Domains field.

That’s it. Simple, eh? Hopefully this will help someone find the solution faster than I did.


Tutorial: Sending email through Gmail

Nowadays I recommend everyone use SendGrid for sending email

An introduction to using Gmail as your Email Service Provider
There are many reasons to consider outsourcing your email. Getting an email into someone’s inbox is not a simple matter these days. There are a million things you need to know to make sure an email actually makes it to the recipient and not into the junk folder. That’s why more and more people are opting to have an email service provider (ESP) take care of sending email for them. With an ESP, you let them keep up with all the blacklists and the whitelists. You can stop worrying about maintaining mail servers and start focusing on the real challenges of your business.

There are a few downsides to using an ESP. First, they are relatively expensive. For a good ESP, you are looking at paying thousands of dollars a year just to send a few emails. Secondly, most ESP’s are focused towards corporate bulk mailers instead of web startups that send lots of customized emails to one person at a time. This makes integration really awkward. Lastly, the ESP’s that have API’s don’t make it easy on you. It’s hard to find an ESP that will integrate nicely with your existing code.

Recently I decided to try using Google Apps for your Domain to send emails from our website. Google Apps offers many of the same benefits as an ESP, but they do it for free and code integration is a non-issue. So far, the results have been great and we’re not losing as many emails to spam filters anymore. I’m not an deliverability expert, but I learned a lot from fighting the email battle this past week. My goal with this tutorial is to share what I learned, so you don’t have to spend a week running all over the web to find it.

The step-by-step tutorial I wish I’d had a week ago:

Set up Google Apps for your domain
Google Apps for your domain comes in a free Standard Edition and a $50/user/year Premier Edition. I took advantage of their offer of one month free with the Premier Edition (mainly for their included tech support), but ultimately I decided that the Standard Edition more than meets my needs. Unless you’ve got some crazy integration needs, Standard Edition will probably do just fine.

Verify your domain
Verifying your domain allows GMail to send emails from your domain instead of on behalf of your domain. You can verify your domain name by either uploading a specific HTML file to your site, or by adding a CNAME record to your DNS records.

Create an account for each email address you want to send from
Google makes it easy to set up multiple accounts with an Excel spreadsheet upload. I’d recommend using the same password for each account to simplify your life and code. If you’re like me, the first thing you will want to do is sign in to each account and forward everything to an email address that you check every day.

Add Google’s MX servers to your DNS records
You will need to add these MX records for Google’s mail servers to your DNS records

Publish an SPF record that includes Google’s SPF record.
A lot of spam filters rely on the Sender Policy Framework (SPF) and Sender ID to decide whether or not to allow your message through or not. An SPF record is simply a DNS record on your server that has a list of IP addresses and domain names that are allowed to send emails from your domain name. In addition to listing the IP addresses of your own servers, you can also include the SPF record from another domain. In this case, we want to include Google’s SPF record to include an up to date list of their mail servers. If you are going to be sending mail from other servers besides google, I recommend you look over this list of common mistakes before you start playing around with your SPF record. You should be particularly careful if you have existing servers that are sending out mail. It’s easy to lock them out with a bad SPF record.

Use a hard fail (for Hotmail’s behalf)
Your SPF record should end with either “~all” or “-all”. The “~all” is a soft fail. This means that servers not listed in your SPF record should be given extra scrutiny. The “-all” is a hard fail. This means that any server that tries to send an email from your domain that is not included on your SPF record will be rejected. Google recommends you use a soft fail (~all) to ensure deliverability, but in practice I found that specifying a hard fail was the only way to get an email into a Hotmail inbox. The hard fail communicates your confidence that your email system is secure. Most spam filters like that.

Submit your SPF record to Hotmail
You need to submit your SPF record to Hotmail to be included in their Sender ID program. Please note that Hotmail caches their records daily. This means if you change something you will need to resubmit your record in order to have your changes take effect immediately.

Subscribe to feedback loops
Feedback loops provide a way for ISP’s to let email senders know when people are clicking the “report spam” button. Google actively subscribes to these feedback loops so they can stop people from sending spam from their mail servers. Google allows you to be informed about these complaints as well. This allows you to quickly remove the complainers from your mailing list. You can get a copy of these complaints by creating email lists for and

Make sure you include a return-path in your email header
Sender ID checks are performed against the purported responsible address (PRA). SPF checks are performed against the return-path (or bounce address). This means you need to make sure you are sending the correct return-path in your email header and not something generic like “noreply@localhost”.

Make sure you are sending quality emails
There are a lot of things you can do to the emails themselves to increase your chances of getting past spam filters. There are a couple “no-brainers”, like making sure you include an unsubscribe link and use valid HTML. There are also a few not-so-obvious things you can do like adding your physical mailing address to the footer of each email. It’s worth taking some time to research what spam filters care about so you can modify your emails accordingly.

Additional Resources:

Please chip in your thoughts / questions. Good luck!


Auto detect a time zone with JavaScript

This blog post will attempt to explain how to automatically detect your user’s time zone using JavaScript. If you’re in a hurry, you can skip directly to the JavaScript timezone detection code on Github.

Previous attempts to solve this problem:

Server side:

Time is not included in an HTTP request. This means that there is no way to get your user’s time zone using a server side scripting language like PHP.

IP address geocoding:

Another method that people have used to address this problem is to geocode your visitors IP address. IP geocoding is what is used when you go to a website and are shown an ad to “meet other singles in Boulder”. Unfortunately, for simply detecting a timezone, IP geo-coding is an expensive way to go. Just check out the prices for Maxmind and ip2location. There’s no way I’m paying for that. I did find a free provider called hostip, but it is worthless as it couldn’t decide whether I live in CA or NC.

With JavaScript:

The common JavaScript that is used to detect a visitor’s timezone is:

var myDate = new Date();

As I started reading up on the getTimezoneOffset code I realized it was too buggy to be used in any critical application. The function returned inconsistent results in different browsers and it never seemed to account for daylight savings time correctly. It quickly became clear that I was going to have to write my own script if I wanted this to work.

How I ended up doing it:

There are basically two things needed to figure out a visitors time zone. First, we need to determine the time offset from Greenwich Mean Time (GMT). This can easily be done by creating two dates (one local, and one in GMT) and comparing the time difference between them:

var rightNow = new Date();
var jan1 = new Date(rightNow.getFullYear(), 0, 1, 0, 0, 0, 0);
var temp = jan1.toGMTString();
var jan2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var std_time_offset = (jan1 - jan2) / (1000 * 60 * 60);

The second thing that you need to know is whether the location observes daylight savings time (DST) or not. Since DST is always observed during the summer, we can compare the time offset between two dates in January, to the time offset between two dates in June. If the offsets are different, then we know that the location observes DST. If the offsets are the same, then we know that the location DOES NOT observe DST.

var june1 = new Date(rightNow.getFullYear(), 6, 1, 0, 0, 0, 0);
temp = june1.toGMTString();
var june2 = new Date(temp.substring(0, temp.lastIndexOf(" ")-1));
var daylight_time_offset = (june1 - june2) / (1000 * 60 * 60);
var dst;
if (std_time_offset == daylight_time_offset) {
    dst = "0"; // daylight savings time is NOT observed
} else {
    dst = "1"; // daylight savings time is observed

Once, I had this code written, the next step was to compile a list of the various time zones around the world along with their opinions on DST. I actually ended up using the list of time zones from Microsoft Windows. It was rather time consuming to compile this list, so I hope you can make use of my work to save yourself some time.

Please let me know if you have any comments, questions or problems with this code. As with anything that I post on this blog, feel free to use this code however you want. Just don’t blame me if it breaks.

Update (06/27/07):

My code wasn’t correctly detecting timezones in the lower hemisphere. I have added hemisphere detection for all our Aussie friends out there. I also fixed a bug in the convert() function that was leaving off the + sign at certain offsets. Thanks Val for pointing this out and helping me with the fix.

Update (10/24/08):

Fixed the bug that Rama and Will pointed out in the comments.

Update (12/22/10):

Jon Nylander has taken my original code and written a a more robust solution. Use his version instead.

How to build great forms

Aristotle said, “We are what we repeatedly do. Excellence, then, is not an act, but a habit”. What many people don’t realize is that excellence isn’t achieved at the macro level. You don’t reach excellence by making one great decision. Instead, you reach excellence through making lots of small decisions consistently well. Here are some simple guidelines to help you improve the quality of your user forms:

1. Do your research

When designing a user interface, take the time to read up on stuff like eyetracking research before you design your forms. For example, did you know that you can reduce your users saccade time from 500ms to 50ms per input field, just by placing form labels above the field instead of left aligning them in another column?

2. Know your user (see #1)

Talk to people! Ask questions. And when all else fails, use common sense. For example, we recently redesigned our date input fields. We were using the popular date picker which allows user’s to select the date from a popup calendar. We thought this feature was cool, until we realized that conference organizers usually register their events 6 or more months in advance – that means they have 6 or more unnecessary clicks just to get to the right month. This didn’t make sense to us, so we went back to the drawing board and opted for regular drop-down date inputs instead.

3. Reduce friction

Every input field that you have people fill out is friction. With every second that you keep people waiting, you increase your likelihood of losing them. You need to do everything you can to reduce this friction. Sit down and evaluate every question in your form design and ask yourself what it would take to get rid of it.

I came across a good example of this yesterday as I was working on the registration form for EventVue. Since our conferences can be located anywhere, we need to collect time zone information. Every website I’ve found requires the user to manually enter their time zone. Why do they do that? Why can’t the browser just tell you which time zone the user is in and automatically select it for them? I spent a day looking into this problem, and sure enough, all it took was some time and some javascript to detect the user’s time zone automagically. So what fields do you have that you could eliminate?

 1 comment