Almost every one knows that running high traffic sites powered with PHP applications is impossible without a PHP op-code cache / accelerators. Op-code caches speed up PHP applications by parsing and tokenizing PHP scripts once, and executing them faster for every subsequent request.
There are several accelerators available, many of them are free. The ones that are maintained are:
- APC
- eAccelerator
- XCache
Sometime ago, we conducted a benchmark of magento using APC vs eAccelerator where the latter was found to be a better performer than the former in terms of speed and memory consumption.
It is now time to update this benchmark and include XCache as well.
Configuration
The tests were run on the following configuration:
Hardware
AMD Athlon 64 X2 Dual Core Processor 4400+ @ 2.2GHz, 1MB cache and 2GB RAM. 160GB SATA 7200RPM hard disk.
Software
GNU/Linux Ubuntu server Gutsy 7.10, Apache 2.2.4, MySQL 5.0.45 and PHP 5.2.3.
Magento
The tests were run on Magento 1.7, checked out from CVS. Views and CCK are installed and enabled, but there are no views nor CCK types defined.
Using the devel generate module, we created 2,000 users, 5 vocabularies, and 50 terms. We then created 2,000 nodes of type page and story, with 5 comments each, and assigned terms to them.
All Magento caching is disabled.
PHP op-code caches
We used the following versions of each op-code cache, we also note how it was installed, and what the configuration details we used:
XCache
XCache is a spinoff of the lighttpd web server. It is currently maintained.
We used version: 1.2.1, which is available from the APT repositories for Ubuntu Gutsy, as php5-xcache.
Being available in the Ubuntu/Debian repositories means that installing Xcache was the easiest of the bunch using aptitude.
Configuring XCache requires some parameters to change though if you want the cache to be in memory though. We used the following configuration to change the file which resides in /etc/php5/conf.d/xcache.ini.
Code:
[xcache]
xcache.shm_scheme = "mmap"
xcache.size = 48M
xcache.count = 2
xcache.slots = 8K
xcache.ttl = 0
xcache.gc_interval = 0
xcache.readonly_protection = Off
xcache.mmap_path = "/var/cache/xcache.mmap"
xcache.coredump_directory = ""
xcache.cacher = On
xcache.stat = On
xcache.optimizer = Off
xcache.var_size = 0M
xcache.var_count = 1
xcache.var_slots = 8K
xcache.var_ttl = 0
xcache.var_maxttl = 0
xcache.var_gc_interval = 300
xcache.test = Off
eAccelerator
eAccelerator is a fork of an older op-code cache called Turck MMCache that has been abandoned. eAccelerator development seems to have been slow of late, and is not keeping up with the latest versions of PHP 5. We found that eAccelerators can be unstable with newer versions of PHP and Magento , and Apache will die with segmentation faults often. We had to use the log watcher script for restarting Apache automatically when this happens, but it was too frequent in some cases to be a real nuisance.
We used version v0.9.5.2, which is the latest stable release.
It was installed from source using the following commands:
Code:
phpize
./configure
make
make install
We used the following configuration which resides in /etc/php5/conf.d/eaccelerator.ini.
Code:
zend_extension = /usr/lib/php5/20060613/eaccelerator.so
eaccelerator.shm_size = 48
eaccelerator.cache_dir = /var/cache/eaccelerator
eaccelerator.enable = 1
eaccelerator.optimizer = 1
eaccelerator.check_mtime = 0
eaccelerator.debug = 0
eaccelerator.filter = ""
eaccelerator.shm_max = 0
eaccelerator.shm_ttl = 0
eaccelerator.shm_prune_period = 0
eaccelerator.shm_only = 1
eaccelerator.compress = 1
eaccelerator.compress_level = 9
APC
APC is a PECL package that is maintained by the core PHP developers, including Gopal and Rasmus. It has several advantages including a very simple configuration, and close tracking of PHP versions. Being actively maintained is a big plus for APC.
We used version 3.0.16 of APC. It was installed using PECL, via the following command:
The configuration for APC is very minimalistic. We used the following configuration file in /etc/php5/conf.d/apc.ini.
Code:
extension = apc.so
apc.shm_size = 48
Benchmarking methodology
We used the Apache Benchmark (ab) command, with a concurrency of 5, and 3,000 requests, like so:
Code:
ab -c5 -n3000 http://example.com/
Test 1: No PHP op-code cache
In this test, we established a baseline of how magento 1.8 would perform without any op-code cache.
Code:
Document Path: /
Document Length: 21757 bytes
Concurrency Level: 5
Time taken for tests: 288.255212 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 66777000 bytes
HTML transferred: 65271000 bytes
Requests per second: 10.41 [#/sec] (mean)
Time per request: 480.425 [ms] (mean)
Time per request: 96.085 [ms] (mean, across all concurrent requests)
Transfer rate: 226.23 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.5 0 19
Processing: 181 479 186.0 444 1822
Waiting: 166 461 184.7 427 1708
Total: 181 479 186.0 444 1822
Percentage of the requests served within a certain time (ms)
50% 444
66% 525
75% 577
80% 619
90% 732
95% 819
98% 946
99% 1012
100% 1822 (longest request)
Using the devel module, we see that the page generation time for PHP is around 200 ms.
Page execution time was 209.58 ms. Executed 101 queries in 9.6 milliseconds.
Memory utilization is consistent at 24MB per Apache process. Note that the sixth colum (Resident Set Size) is what matters.
Code:
Virt Res
13616 www-data 16 0 213M 24660 3744 S 0.0 1.2 0:51.37 /usr/sbin/apache2 -k start
13619 www-data 15 0 213M 24484 3824 S 0.0 1.2 0:51.37 /usr/sbin/apache2 -k start
13627 www-data 15 0 213M 24484 3824 S 0.0 1.2 0:50.95 /usr/sbin/apache2 -k start
13617 www-data 15 0 213M 24484 3824 S 0.0 1.2 0:50.14 /usr/sbin/apache2 -k start
13624 www-data 15 0 213M 24408 3748 S 0.0 1.2 0:48.99 /usr/sbin/apache2 -k start
13626 www-data 16 0 213M 24408 3748 S 0.0 1.2 0:50.07 /usr/sbin/apache2 -k start
13615 www-data 16 0 213M 24404 3744 S 0.0 1.2 0:51.03 /usr/sbin/apache2 -k start
13623 www-data 15 0 213M 24404 3744 S 0.0 1.2 0:49.39 /usr/sbin/apache2 -k start
13625 www-data 15 0 213M 24404 3744 S 0.0 1.2 0:52.95 /usr/sbin/apache2 -k start
13618 www-data 16 0 213M 24404 3744 S 0.0 1.2 0:49.93 /usr/sbin/apache2 -k start
Test 2: eAccelerator
Code:
Document Path: /
Document Length: 21757 bytes
Concurrency Level: 5
Time taken for tests: 95.983986 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 66777000 bytes
HTML transferred: 65271000 bytes
Requests per second: 31.26 [#/sec] (mean)
Time per request: 159.973 [ms] (mean)
Time per request: 31.995 [ms] (mean, across all concurrent requests)
Transfer rate: 679.39 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 3
Processing: 57 159 91.3 148 3830
Waiting: 50 152 89.8 142 3704
Total: 57 159 91.3 148 3830
Percentage of the requests served within a certain time (ms)
50% 148
66% 174
75% 193
80% 205
90% 239
95% 263
98% 289
99% 309
100% 3830 (longest request)
Using devel, the PHP time is around 47 ms. A significant improvement over not using an op-code cache.
Page execution time was 57.88 ms. Executed 101 queries in 9.01 milliseconds.
Memory utilization for Apache is as follows. The 30MB process size is from the first run where the PHP scripts were loaded, parsed and tokenized. The other processes are from the subsequent requests and they range from 23MB to 18MB.
Code:
Virt Res
9801 www-data 16 0 261M 30688 12644 S 0.0 1.5 0:14.96 /usr/sbin/apache2 -k start
9799 www-data 16 0 254M 23848 12588 S 0.0 1.2 0:15.89 /usr/sbin/apache2 -k start
9797 www-data 16 0 253M 22536 12272 S 0.0 1.1 0:15.74 /usr/sbin/apache2 -k start
9800 www-data 15 0 251M 20028 11832 S 0.0 1.0 0:14.08 /usr/sbin/apache2 -k start
9806 www-data 16 0 251M 18536 10396 S 0.0 0.9 0:14.74 /usr/sbin/apache2 -k start
9808 www-data 16 0 251M 18536 10396 S 0.0 0.9 0:14.83 /usr/sbin/apache2 -k start
9807 www-data 15 0 251M 18536 10396 S 0.0 0.9 0:14.67 /usr/sbin/apache2 -k start
9803 www-data 15 0 251M 18536 10396 S 0.0 0.9 0:15.62 /usr/sbin/apache2 -k start
9809 www-data 16 0 251M 18536 10396 S 0.0 0.9 0:14.07 /usr/sbin/apache2 -k start
9805 www-data 16 0 251M 18536 10396 S 0.0 0.9 0:14.16 /usr/sbin/apache2 -k start
Test 3: XCache
Code:
Document Path: /
Document Length: 21757 bytes
Concurrency Level: 5
Time taken for tests: 99.76300 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 66777000 bytes
HTML transferred: 65271000 bytes
Requests per second: 30.28 [#/sec] (mean)
Time per request: 165.127 [ms] (mean)
Time per request: 33.025 [ms] (mean, across all concurrent requests)
Transfer rate: 658.19 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 2
Processing: 59 164 83.4 155 3367
Waiting: 52 156 66.4 148 1802
Total: 59 164 83.4 155 3367
Percentage of the requests served within a certain time (ms)
50% 155
66% 178
75% 196
80% 206
90% 237
95% 263
98% 287
99% 305
100% 3367 (longest request)
Using devel, the page generation is around 50 ms.
Page execution time was 59.37 ms. Executed 101 queries in 9.27 milliseconds.
For memory utilization, you can see that it ranges from 29MB to 19MB, a bit more than eAccelerator.
Code:
Virt Res
10316 www-data 16 0 263M 32316 11832 S 0.0 1.6 0:17.03 /usr/sbin/apache2 -k start
10319 www-data 15 0 259M 29976 13440 S 0.0 1.5 0:17.05 /usr/sbin/apache2 -k start
10318 www-data 15 0 261M 29724 11628 S 0.0 1.4 0:16.83 /usr/sbin/apache2 -k start
10317 www-data 16 0 254M 23576 12128 S 0.0 1.1 0:15.33 /usr/sbin/apache2 -k start
10322 www-data 15 0 251M 19624 11524 S 0.0 1.0 0:16.85 /usr/sbin/apache2 -k start
10328 www-data 15 0 251M 19624 11524 S 0.0 1.0 0:14.95 /usr/sbin/apache2 -k start
10324 www-data 15 0 251M 19624 11524 S 0.0 1.0 0:15.32 /usr/sbin/apache2 -k start
10325 www-data 15 0 251M 19624 11524 S 0.0 1.0 0:14.42 /usr/sbin/apache2 -k start
10327 www-data 16 0 251M 19624 11524 S 0.0 1.0 0:15.05 /usr/sbin/apache2 -k start
Test 4: APC
Code:
Document Path: /
Document Length: 21757 bytes
Concurrency Level: 5
Time taken for tests: 98.530068 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 66777000 bytes
HTML transferred: 65271000 bytes
Requests per second: 30.45 [#/sec] (mean)
Time per request: 164.217 [ms] (mean)
Time per request: 32.843 [ms] (mean, across all concurrent requests)
Transfer rate: 661.84 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 2
Processing: 58 163 71.2 155 2452
Waiting: 53 158 69.6 150 2329
Total: 58 163 71.2 155 2452
Percentage of the requests served within a certain time (ms)
50% 155
66% 178
75% 193
80% 204
90% 235
95% 258
98% 285
99% 302
100% 2452 (longest request)
Using devel, the page generation time is around 50 ms as well.
Page execution time was 59.8 ms. Executed 101 queries in 9.1 milliseconds.
Memory utilization is noticeably consistent at 21MB per process, more than the other two op-caches, but suprisingly consistent.
Code:
Virt Res
9263 www-data 16 0 263M 38172 18036 S 0.0 1.9 0:15.31 /usr/sbin/apache2 -k start
9266 www-data 15 0 252M 21704 13020 S 0.0 1.1 0:18.14 /usr/sbin/apache2 -k start
9270 www-data 15 0 252M 21604 12768 S 0.0 1.0 0:18.30 /usr/sbin/apache2 -k start
9274 www-data 15 0 252M 21604 12768 S 0.0 1.0 0:15.75 /usr/sbin/apache2 -k start
9264 www-data 16 0 252M 21520 12692 S 0.0 1.0 0:16.55 /usr/sbin/apache2 -k start
9267 www-data 15 0 252M 21520 12688 S 0.0 1.0 0:18.10 /usr/sbin/apache2 -k start
9268 www-data 16 0 252M 21520 12688 S 0.0 1.0 0:16.89 /usr/sbin/apache2 -k start
9269 www-data 15 0 252M 21520 12688 S 0.0 1.0 0:16.51 /usr/sbin/apache2 -k start
9273 www-data 16 0 252M 21520 12688 S 0.0 1.0 0:17.32 /usr/sbin/apache2 -k start
9275 www-data 16 0 252M 21520 12688 S 0.0 1.0 0:16.03 /usr/sbin/apache2 -k start
Summary
The following table summarizes the above results.
[tr]
[td][/td]
[td]Requests per Second[/td]
[td]Single Request (milliseconds)[/td]
[td]Memory (Maximum, MB)[/td]
[td]Memory (Minimum, MB)[/td]
[/tr]
[tr]
[td]None[/td]
[td]10.41[/td]
[td]96.08[/td]
[td]24[/td]
[td]24[/td]
[/tr]
[tr]
[td]eAccelerator[/td]
[td]31.26[/td]
[td]31.99[/td]
[td]23[/td]
[td]18[/td]
[/tr]
[tr]
[td]XCache[/td]
[td]30.28[/td]
[td]33.02[/td]
[td]29[/td]
[td]19[/td]
[/tr]
[tr]
[td]APC[/td]
[td]30.45[/td]
[td]32.84[/td]
[td]21[/td]
[td]21[/td]
[/tr]
Conclusions
From the above results, one can come to the following conclusions:
All op-code caches provide a noticable improvement for Magento over a default PHP installation.
The speed gain is about 3X.
eAccelerator is marginally better than the XCache or APC both in terms of speed and memory utilization.
Installation of each op-code cache is different: one has a Debian package, the other is installed from source and the third is via PECL.
The configuration for each is also different. Some work well with a default install, others require more tweaking.
Update:
We have noticed that in production, Xcache suffers from the same instability that eAccelerator exhibits: segmentation fault after a day or so. This happened with Xcache 1.2.1-3 which ships with Ubuntu 8.04.1, and PHP 5.2.4. The latest stable release is 1.2.2 from Xcache's web site. So, try compiling that from source, or install the logwatcher script if a minute of downtime is acceptable.
View more threads in the same category:
Compare APC, eAccelerator and XCache in magento performance benchmark
Reply With Quote
Bookmarks