Dabr migration and Apache
After a recent purchase of VPS hosting from WebFusion, I wanted to move my high-traffic site dabr.co.uk onto it. With the site exceeding one million daily page views, I really had to be careful about how I’d move the site to it’s new home.
10%
Using the “Hostnames” report within Google Analytics, I was able to identify that 10% of users access the site via the m subdomain and 40% use www – the rest not using any subdomain at all. This was a real stroke of luck as it allowed me to test the new server with a controllable percentage of the total traffic, simply by updating the DNS records.
When given 10% load the VPS handled just fine, so after two days I moved on and tried it with 50%.
50%
A second report from Google Analytics also helped tremendously: hourly visits. With about 45% of hits coming from Indonesia (yes, really), I needed to know what time was the best window to try any drastic changes to the servers. The graph clearly shows that the longest quiet period I get is between 6pm and 9pm GMT – when most of Indonesia is asleep and web traffic drops in half:
On the first few occasions that I did try 50% of the load pointing at the new server – it buckled and almost completely fell over. A chat with Nick, a friend from Heavy Sumo, led me to explore the Apache configuration – and what a bizarre experience that was.
Here’s the settings that were used by default on the server, compared with the Apache defaults and the settings on the old server:
| Setting | WebFusion VPS | Apache default | Old server |
|---|---|---|---|
| StartServers | 1 | 5 | 8 |
| MinSpareServers | 1 | 5 | 5 |
| MaxSpareServers | 5 | 10 | 20 |
| MaxClients | 10 | 256 | 256 |
| MaxRequestsPerChild | 0 | 1000 | 4000 |
Notice the difference?
I have no idea why WebFusion would have their defaults so low. Changing these to match the settings of the old server had a significant boost on performance.
I left 50% traffic on the new server for a couple of weeks, and tried a couple of times to add a bit more, but users quickly complained about the site slowing down. The slow down matched up to the Indonesian peak time around 2pm GMT, so I rolled back the DNS to point to the old server again.
90%
This stage required a little further exploration of Apache performance tuning articles. One interesting fact I discovered was that Apache has a hard-coded MaxClients limit of 256, which you can only raise by recompiling Apache – so that wasn’t particularly an ideal option.
The Apache Performance Tuning article from devside.net was the answer to my problems. It suggested a number of different solutions, two of which were to adjust Keep Alive and Timeout settings.
| Setting | Webfusion VPS | Devside |
|---|---|---|
| Timeout | 300 | 45 |
| MaxKeepAliveRequests | 100 | 80 |
| KeepAliveTimeout | 15 | 2 |
These changes, combined with the forking settings above, have made the server run much more smoothly. In fact, it’s now taking 100% of the traffic and people are noticing an improvement in speed.
100%
With the new server running well for a month, I then asked the previous host to delete the old server. Hopefully the new server should last a while before I need to consider load balancing to a second server.