Your blog/website being popular, more visitors come, you would find your website is slow and the servers are not able to handle that much of requests. However, there are some steps can optimize your website quickly. This tricks applied to your website/blog running on Drupal or WordPress or Zend Framework or any PHP Framework ...... etc.
1. Benchmarking your web server
To optimize your web server performance, firstly, we have to perform a proper assessment. Here, I am going to use "Apache Benchmark". Using this tools to query a specific page of your website and measure the response. To do so, you can execute the following command line from the server you are testing since it has low CPU and RAM footprint.
n = number of page requests
c = concurrent connections
The only parameter need to play around is "c", since "n" is the number that needs to be large enough to provde stable results. The aims of this test is to test the website with various "c" values. The "c" value should start with small number and increase it until the values of "Requests per second" starts dropping.
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 45.29 [#/sec] (mean)
ab -n 1000 -c 240 http://mywebsite.com/ | grep 'Requests per second'
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 46.91 [#/sec] (mean)
ab -n 1000 -c 260 http://mywebsite.com/ | grep 'Requests per second'
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 8.55 [#/sec] (mean)
ab -n 1000 -c 280 http://mywebsite.com/ | grep 'Requests per second'
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 2.21 [#/sec] (mean)
2. Reduce memory footprint until paging stops
From the above result, we can see the requests per second drop suddenly. It is bceause there are too much requests which overload the memory and the system start using the paging file. The similar effects happen in Windows or Mac OSX when there are too many applications opened.
total used free shared buffers cached
Mem: 4011 1481 2530 0 45 824
-/+ buffers/cache: 611 3400
Swap: 8191 2145 8191
By execute the "free -m" command, we can see how much of swap file is used. Here, the server has 2145MB of swap file which is captured when the Apache Benchmark was run with largest "c" value.
$ sudo swapoff -a
$ sudo swapon -a
$ free -m
total used free shared buffers cached
Mem: 4011 1481 2530 0 45 824
-/+ buffers/cache: 499 3512
Swap: 8191 0 8191
Here, we are going to redo the test above with "c" values that are close to the one where we see drop on performance. And here are going to check the paging file after each request:
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 45.93 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 0 8191
ab -n 1000 -c 240 http://mywebsite.com/ | grep 'Requests per second'
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 40.31 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 312 8191
ab -n 1000 -c 250 http://mywebsite.com/ | grep 'Requests per second'
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Requests per second: 12.27 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 1902 8191
From the above result, we can see the rate of request drops significantly when there are 250 concurrent requests.
3. Configurate the max number of connections
By default, both Apache and MySQL are configurated to accept 150 connections. For most PHP applications (e.g., Drupal) will open one connection per thread. In order to know how much memory required to handle one request, you can execute TOP command:
Tasks: 136 total, 1 running, 134 sleeping, 0 stopped, 1 zombie
Cpu(s): 8.0%us, 5.7%sy, 0.0%ni, 80.3%id, 1.3%wa, 0.0%hi, 4.7%si, 0.0%st
Mem: 4108192k total, 1787064k used, 2321128k free, 46520k buffers
Swap: 8388604k total, 0k used, 8388604k free, 861772k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2005 root 20 0 11844 6752 1544 S 3.0 0.3 0:00.09 ab
745 mysql 20 0 147m 24m 6056 S 0.3 1.2 0:06.23 mysqld
1715 www-data 20 0 62088 27m 4112 S 0.3 1.4 0:03.78 apache2
1741 www-data 20 0 59628 25m 4844 S 0.3 1.3 0:06.49 apache2
1784 www-data 20 0 61840 27m 4128 S 0.3 1.4 0:01.27 apache2
1820 www-data 20 0 59228 24m 4156 S 0.3 1.2 0:01.60 apache2
1942 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.05 apache2
1945 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.04 apache2
1946 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.02 apache2
1948 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.03 apache2
1949 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.01 apache2
1957 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.01 apache2
1965 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.04 apache2
1966 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.06 apache2
1967 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.02 apache2
1968 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.06 apache2
1972 www-data 20 0 43268 6288 1360 S 0.3 0.3 0:00.02 apache2
By monitoring the result above, we can estimate how much memory is used by Apache, by averaging the value of RES (resident non-swapped physical memory) used by the processes. Here the average value of RES is about 62MB.
Also, you can have a look the amount of free memory availble when the server is not under load:
Mem: 4108192k total, 1388132k used, 2720060k free, 46680k buffers
Swap: 8388604k total, 0k used, 8388604k free, 866208k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1939 root 20 0 2548 1196 904 R 0.3 0.1 0:00.62 top
1 root 20 0 2656 1532 1140 S 0.0 0.1 0:00.38 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0
6 root 20 0 0 0 0 S 0.0 0.0 0:00.23 events/0
7 root 20 0 0 0 0 S 0.0 0.0 0:00.00 cpuset
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khelper
9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 async/mgr
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 pm
11 root 20 0 0 0 0 S 0.0 0.0 0:00.01 sync_supers
12 root 20 0 0 0 0 S 0.0 0.0 0:00.02 bdi-default
13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kintegrityd/0
14 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kblockd/0
15 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpid
16 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpi_notify
17 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kacpi_hotplug
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 ata/0
From above, we know that there are 2720060KB (2656MB) free memory available.
Based on above value, we can use a simple calculation to find out the optimal numbers of concurrent users:
c = 2656MB / 62MB
c = 43 (close to the 40 found above)
We can concluded that the server starts melting when there are more than 43 concurrent users. And here, we can reduce the number of clients from 150 to 40 in both Apache & MySQL configuration.
Once the above changes are adjusted, we can re-run the benchmark again:
Requests per second: 44.43 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 0 8191
$ ab -n 1000 -c 40 http://mywebsite.com/ | grep ‘Requests per second’
Requests per second: 45.11 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 0 8191
$ ab -n 1000 -c 60 http://mywebsite.com/ | grep ‘Requests per second’
Requests per second: 43.27 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 0 8191
$ ab -n 1000 -c 80 http://mywebsite.com/ | grep ‘Requests per second’
Requests per second: 44.59 [#/sec] (mean)
$ free -m | grep Swap
Swap: 8191 0 8191
As you may notice, it seems like that the Apache able to answer more than 40 concurrent request?? Actually, the Apache only process 40 concurrent requests. Any request beyond the max concurrent number are put in a queue and he client may experience some delay.
4. Performance of MySQL depends on RAM
In fact, MySQL does not consume much CPU. MySQL will consume 10 - 25% of CPU if the server well configurated. However, MySQL is very RAM consuming. It is common to see MySQL use close to 50% of the RAM available even MySQL is having its caching and threads optimized.
To optimize MySQL performance, the rule is to give as much RAM as possible to MySQL.
5. Performance of Apache depends on CPU
On Drupal, WordPress or others website based on PHP Framework, Apache spends most of its time to execute PHP code. Apache divide the available CPU across each one to answer request. Hence, CPU is the key resources of Apache.