Google bot crawl stats for my site
This screen shot is from my Google webmaster tools account. The first two graphs show the number of pages crawled and the number of kilobytes downloaded per day. The last graph shows the time that Google bots needed to download pages.
You can see that Google has been crawling my site as much as ever, but since I tuned my Raspberry Pi server cluster, the time spent downloading pages has plummeted.
I’ve tuned Apache on my Raspberry Pi cluster. I’ve removed some unused modules, and increased the number of server processes. Tests with siege show that my cluster can handle more concurrent users, and response times are shorter. These were simple changes to make, and they seem to have made a big difference. The tuned server cluster can now handle 940 transactions per second according to siege.