Anatomy of a Fireballing

This blog does OK. I get a couple of hundred visits a day, including a steady double-digit daily amount coming from HDRsoft for my HDR posts and the occasional spike towards the thousand whenever someone new discovers the Munchy Box and passes it to their friends. However, just a couple of days ago, I was inundated with visits coming from Daring Fireball. Oh crap.

Now, this isn’t the first time I’ve had a WordPress blog spiked by a tsunami of visits: This very blog has had three major influxes of traffic from the B3ta newsletter, and the funny-named myqu.im made B3ta’s funny name corner. What I learned from these previous successes was that WordPress will fall over every time, taking MySQL out first and cascading Apache as collateral damage. While the Munchy Box post was getting around 4-5,000 visits every day for about a month, giving me plenty of time to learn how to stop my server from breaking, Daring Fireball gave me about one new visit a second, for over an hour.

jearle@cat5# sudo grep "GET /200" statistics/logs/access_log.processed | \
    awk '/daring/{print $4" "$5}' | head -1
[30/Nov/2009:00:29:37 +0000]
jearle@cat5# sudo grep "GET /200" statistics/logs/access_log.processed | \
    grep -c "30/Nov/2009:00"
1503

First visit at half past midnight and 1500 visits in the first half-hour after that. That’s what a site needs to be able to withstand if you get a link posted on Daring Fireball. Frankly, on a standard, average dedicated server, WordPress cannot handle that amount of traffic without help. Luckily, help is to hand in the form of Donncha O Caoimh‘s superb wp-super-cache, a WordPress plugin that creates a static cache of your site so it doesn’t need to spend over a second generating a page for each visit.

jearle@cat5# curl -s blog.23x.net/200 | tail -3
<!-- Dynamic page generated in 1.545 seconds. -->
<!-- Cached page generated by WP-Super-Cache on 2009-12-03 00:14:07 -->
<!-- super cache -->

To put it simply, if you’re running a WordPress blog, you need to be using wp-super-cache. One day, someone will tell people about stuff you’re saying and you want your site to still be there when the people that listen to them turn up on your doorstep. When I first set up myqu.im, I didn’t install wp-super-cache and had to hard-reset the server from the serial console to regain control when the b3tans overloaded it in 5 minutes. Don’t be as dumb as I was. This time, my CPU was 95%+ IDLE and MySQL didn’t even show in top.

Anyway, enough about that, let’s look at what Daring Fireball readers use. Thanks to Google Anal for the stats.

First up is the mix of browsers. Here’s a typical lazy Sunday’s stats, the day before a Fireballing. Safari beats IE, but languishes behind Firefox.

This is a pie chart of the browsers used by the 13,000 visits Daring Fireball brought me. Over half use Safari.

As Daring Fireball is mostly an Apple commentary site, we would expect the OS bias to fall heavily towards the fruited-ones like myself. My usual crowd uses mostly Windows, but is already 33% Mac.

Gruber’s readers are almost three-quarters Mac-based, and shockingly, even on a work-day, there are more iPhone users than Windows users.

Now, let’s not forget these stats are just numbers and numbers don’t read articles, but still, more iPhones than PCs? Any questions? I’ve got the raw data here if anyone needs more info …