Magento Expert Forum - Improve your Magento experience

Results 1 to 2 of 2

Compare APC, eAccelerator and XCache in magento performance benchmark

  1. #1
    Junior Member jaredovi's Avatar
    Join Date
    Mar 2013
    Posts
    68
    Thanks
    2
    Thanked 14 Times in 11 Posts

    Default Compare APC, eAccelerator and XCache in magento performance benchmark

    Almost every one knows that running high traffic sites powered with PHP applications is impossible without a PHP op-code cache / accelerators. Op-code caches speed up PHP applications by parsing and tokenizing PHP scripts once, and executing them faster for every subsequent request.

    There are several accelerators available, many of them are free. The ones that are maintained are:


    1. APC
    2. eAccelerator
    3. XCache



    Sometime ago, we conducted a benchmark of magento using APC vs eAccelerator where the latter was found to be a better performer than the former in terms of speed and memory consumption.

    It is now time to update this benchmark and include XCache as well.

    Configuration

    The tests were run on the following configuration:

    Hardware

    AMD Athlon 64 X2 Dual Core Processor 4400+ @ 2.2GHz, 1MB cache and 2GB RAM. 160GB SATA 7200RPM hard disk.

    Software

    GNU/Linux Ubuntu server Gutsy 7.10, Apache 2.2.4, MySQL 5.0.45 and PHP 5.2.3.

    Magento


    The tests were run on Magento 1.7, checked out from CVS. Views and CCK are installed and enabled, but there are no views nor CCK types defined.

    Using the devel generate module, we created 2,000 users, 5 vocabularies, and 50 terms. We then created 2,000 nodes of type page and story, with 5 comments each, and assigned terms to them.

    All Magento caching is disabled.

    PHP op-code caches

    We used the following versions of each op-code cache, we also note how it was installed, and what the configuration details we used:

    XCache


    XCache is a spinoff of the lighttpd web server. It is currently maintained.

    We used version: 1.2.1, which is available from the APT repositories for Ubuntu Gutsy, as php5-xcache.

    Being available in the Ubuntu/Debian repositories means that installing Xcache was the easiest of the bunch using aptitude.

    Configuring XCache requires some parameters to change though if you want the cache to be in memory though. We used the following configuration to change the file which resides in /etc/php5/conf.d/xcache.ini.


    Code:
    [xcache]
    xcache.shm_scheme =    "mmap"
    xcache.size =        48M
    xcache.count =         2
    xcache.slots =        8K
    xcache.ttl  =         0
    xcache.gc_interval =      0
    xcache.readonly_protection = Off
    xcache.mmap_path =      "/var/cache/xcache.mmap"
    xcache.coredump_directory = ""
    xcache.cacher =       On
    xcache.stat  =       On
    xcache.optimizer =      Off
    xcache.var_size =      0M
    xcache.var_count =       1
    xcache.var_slots =      8K
    xcache.var_ttl  =       0
    xcache.var_maxttl  =     0
    xcache.var_gc_interval =   300
    xcache.test =        Off

    eAccelerator


    eAccelerator is a fork of an older op-code cache called Turck MMCache that has been abandoned. eAccelerator development seems to have been slow of late, and is not keeping up with the latest versions of PHP 5. We found that eAccelerators can be unstable with newer versions of PHP and Magento , and Apache will die with segmentation faults often. We had to use the log watcher script for restarting Apache automatically when this happens, but it was too frequent in some cases to be a real nuisance.

    We used version v0.9.5.2, which is the latest stable release.

    It was installed from source using the following commands:

    Code:
    phpize
    ./configure
    make
    make install
    We used the following configuration which resides in /etc/php5/conf.d/eaccelerator.ini.


    Code:
    zend_extension         = /usr/lib/php5/20060613/eaccelerator.so
    eaccelerator.shm_size      = 48
    eaccelerator.cache_dir     = /var/cache/eaccelerator
    eaccelerator.enable       = 1
    eaccelerator.optimizer     = 1
    eaccelerator.check_mtime    = 0
    eaccelerator.debug       = 0
    eaccelerator.filter       = ""
    eaccelerator.shm_max      = 0
    eaccelerator.shm_ttl      = 0
    eaccelerator.shm_prune_period  = 0
    eaccelerator.shm_only      = 1
    eaccelerator.compress      = 1
    eaccelerator.compress_level   = 9

    APC


    APC is a PECL package that is maintained by the core PHP developers, including Gopal and Rasmus. It has several advantages including a very simple configuration, and close tracking of PHP versions. Being actively maintained is a big plus for APC.

    We used version 3.0.16 of APC. It was installed using PECL, via the following command:


    Code:
    pecl install apc
    The configuration for APC is very minimalistic. We used the following configuration file in /etc/php5/conf.d/apc.ini.


    Code:
    extension         = apc.so
    apc.shm_size       = 48

    Benchmarking methodology


    We used the Apache Benchmark (ab) command, with a concurrency of 5, and 3,000 requests, like so:


    Code:
    ab -c5 -n3000 http://example.com/

    Test 1: No PHP op-code cache



    In this test, we established a baseline of how magento 1.8 would perform without any op-code cache.

    Code:
    Document Path:     /
    Document Length:    21757 bytes
    Concurrency Level:   5
    Time taken for tests:  288.255212 seconds
    Complete requests:   3000
    Failed requests:    0
    Write errors:      0
    Total transferred:   66777000 bytes
    HTML transferred:    65271000 bytes
    Requests per second:  10.41 [#/sec] (mean)
    Time per request:    480.425 [ms] (mean)
    Time per request:    96.085 [ms] (mean, across all concurrent requests)
    Transfer rate:     226.23 [Kbytes/sec] received
    Connection Times (ms)
    min mean[+/-sd] median  max
    Connect:    0  0  0.5   0   19
    Processing:  181 479 186.0  444  1822
    Waiting:   166 461 184.7  427  1708
    Total:    181 479 186.0  444  1822
    Percentage of the requests served within a certain time (ms)
    50%  444
    66%  525
    75%  577
    80%  619
    90%  732
    95%  819
    98%  946
    99%  1012
    100%  1822 (longest request)

    Using the devel module, we see that the page generation time for PHP is around 200 ms.

    Page execution time was 209.58 ms. Executed 101 queries in 9.6 milliseconds.

    Memory utilization is consistent at 24MB per Apache process. Note that the sixth colum (Resident Set Size) is what matters.

    Code:
    Virt Res
    13616 www-data 16  0 213M 24660 3744 S 0.0 1.2 0:51.37 /usr/sbin/apache2 -k start
    13619 www-data 15  0 213M 24484 3824 S 0.0 1.2 0:51.37 /usr/sbin/apache2 -k start
    13627 www-data 15  0 213M 24484 3824 S 0.0 1.2 0:50.95 /usr/sbin/apache2 -k start
    13617 www-data 15  0 213M 24484 3824 S 0.0 1.2 0:50.14 /usr/sbin/apache2 -k start
    13624 www-data 15  0 213M 24408 3748 S 0.0 1.2 0:48.99 /usr/sbin/apache2 -k start
    13626 www-data 16  0 213M 24408 3748 S 0.0 1.2 0:50.07 /usr/sbin/apache2 -k start
    13615 www-data 16  0 213M 24404 3744 S 0.0 1.2 0:51.03 /usr/sbin/apache2 -k start
    13623 www-data 15  0 213M 24404 3744 S 0.0 1.2 0:49.39 /usr/sbin/apache2 -k start
    13625 www-data 15  0 213M 24404 3744 S 0.0 1.2 0:52.95 /usr/sbin/apache2 -k start
    13618 www-data 16  0 213M 24404 3744 S 0.0 1.2 0:49.93 /usr/sbin/apache2 -k start

    Test 2: eAccelerator




    Code:
    Document Path:     /
    Document Length:    21757 bytes
    Concurrency Level:   5
    Time taken for tests:  95.983986 seconds
    Complete requests:   3000
    Failed requests:    0
    Write errors:      0
    Total transferred:   66777000 bytes
    HTML transferred:    65271000 bytes
    Requests per second:  31.26 [#/sec] (mean)
    Time per request:    159.973 [ms] (mean)
    Time per request:    31.995 [ms] (mean, across all concurrent requests)
    Transfer rate:     679.39 [Kbytes/sec] received
    Connection Times (ms)
    min mean[+/-sd] median  max
    Connect:    0  0  0.1   0    3
    Processing:  57 159 91.3  148  3830
    Waiting:    50 152 89.8  142  3704
    Total:     57 159 91.3  148  3830
    Percentage of the requests served within a certain time (ms)
    50%  148
    66%  174
    75%  193
    80%  205
    90%  239
    95%  263
    98%  289
    99%  309
    100%  3830 (longest request)

    Using devel, the PHP time is around 47 ms. A significant improvement over not using an op-code cache.

    Page execution time was 57.88 ms. Executed 101 queries in 9.01 milliseconds.

    Memory utilization for Apache is as follows. The 30MB process size is from the first run where the PHP scripts were loaded, parsed and tokenized. The other processes are from the subsequent requests and they range from 23MB to 18MB.


    Code:
    Virt Res
    9801 www-data 16  0 261M 30688 12644 S 0.0 1.5 0:14.96 /usr/sbin/apache2 -k start
    9799 www-data 16  0 254M 23848 12588 S 0.0 1.2 0:15.89 /usr/sbin/apache2 -k start
    9797 www-data 16  0 253M 22536 12272 S 0.0 1.1 0:15.74 /usr/sbin/apache2 -k start
    9800 www-data 15  0 251M 20028 11832 S 0.0 1.0 0:14.08 /usr/sbin/apache2 -k start
    9806 www-data 16  0 251M 18536 10396 S 0.0 0.9 0:14.74 /usr/sbin/apache2 -k start
    9808 www-data 16  0 251M 18536 10396 S 0.0 0.9 0:14.83 /usr/sbin/apache2 -k start
    9807 www-data 15  0 251M 18536 10396 S 0.0 0.9 0:14.67 /usr/sbin/apache2 -k start
    9803 www-data 15  0 251M 18536 10396 S 0.0 0.9 0:15.62 /usr/sbin/apache2 -k start
    9809 www-data 16  0 251M 18536 10396 S 0.0 0.9 0:14.07 /usr/sbin/apache2 -k start
    9805 www-data 16  0 251M 18536 10396 S 0.0 0.9 0:14.16 /usr/sbin/apache2 -k start

    Test 3: XCache




    Code:
    Document Path:     /
    Document Length:    21757 bytes
    Concurrency Level:   5
    Time taken for tests:  99.76300 seconds
    Complete requests:   3000
    Failed requests:    0
    Write errors:      0
    Total transferred:   66777000 bytes
    HTML transferred:    65271000 bytes
    Requests per second:  30.28 [#/sec] (mean)
    Time per request:    165.127 [ms] (mean)
    Time per request:    33.025 [ms] (mean, across all concurrent requests)
    Transfer rate:     658.19 [Kbytes/sec] received
    Connection Times (ms)
    min mean[+/-sd] median  max
    Connect:    0  0  0.0   0    2
    Processing:  59 164 83.4  155  3367
    Waiting:    52 156 66.4  148  1802
    Total:     59 164 83.4  155  3367
    Percentage of the requests served within a certain time (ms)
    50%  155
    66%  178
    75%  196
    80%  206
    90%  237
    95%  263
    98%  287
    99%  305
    100%  3367 (longest request)
    Using devel, the page generation is around 50 ms.

    Page execution time was 59.37 ms. Executed 101 queries in 9.27 milliseconds.

    For memory utilization, you can see that it ranges from 29MB to 19MB, a bit more than eAccelerator.

    Code:
    Virt Res
    10316 www-data 16  0 263M 32316 11832 S 0.0 1.6 0:17.03 /usr/sbin/apache2 -k start
    10319 www-data 15  0 259M 29976 13440 S 0.0 1.5 0:17.05 /usr/sbin/apache2 -k start
    10318 www-data 15  0 261M 29724 11628 S 0.0 1.4 0:16.83 /usr/sbin/apache2 -k start
    10317 www-data 16  0 254M 23576 12128 S 0.0 1.1 0:15.33 /usr/sbin/apache2 -k start
    10322 www-data 15  0 251M 19624 11524 S 0.0 1.0 0:16.85 /usr/sbin/apache2 -k start
    10328 www-data 15  0 251M 19624 11524 S 0.0 1.0 0:14.95 /usr/sbin/apache2 -k start
    10324 www-data 15  0 251M 19624 11524 S 0.0 1.0 0:15.32 /usr/sbin/apache2 -k start
    10325 www-data 15  0 251M 19624 11524 S 0.0 1.0 0:14.42 /usr/sbin/apache2 -k start
    10327 www-data 16  0 251M 19624 11524 S 0.0 1.0 0:15.05 /usr/sbin/apache2 -k start

    Test 4: APC




    Code:
    Document Path:     /
    Document Length:    21757 bytes
    Concurrency Level:   5
    Time taken for tests:  98.530068 seconds
    Complete requests:   3000
    Failed requests:    0
    Write errors:      0
    Total transferred:   66777000 bytes
    HTML transferred:    65271000 bytes
    Requests per second:  30.45 [#/sec] (mean)
    Time per request:    164.217 [ms] (mean)
    Time per request:    32.843 [ms] (mean, across all concurrent requests)
    Transfer rate:     661.84 [Kbytes/sec] received
    Connection Times (ms)
    min mean[+/-sd] median  max
    Connect:    0  0  0.0   0    2
    Processing:  58 163 71.2  155  2452
    Waiting:    53 158 69.6  150  2329
    Total:     58 163 71.2  155  2452
    Percentage of the requests served within a certain time (ms)
    50%  155
    66%  178
    75%  193
    80%  204
    90%  235
    95%  258
    98%  285
    99%  302
    100%  2452 (longest request)

    Using devel, the page generation time is around 50 ms as well.


    Page execution time was 59.8 ms. Executed 101 queries in 9.1 milliseconds.


    Memory utilization is noticeably consistent at 21MB per process, more than the other two op-caches, but suprisingly consistent.


    Code:
    Virt Res
    9263 www-data 16  0 263M 38172 18036 S 0.0 1.9 0:15.31 /usr/sbin/apache2 -k start
    9266 www-data 15  0 252M 21704 13020 S 0.0 1.1 0:18.14 /usr/sbin/apache2 -k start
    9270 www-data 15  0 252M 21604 12768 S 0.0 1.0 0:18.30 /usr/sbin/apache2 -k start
    9274 www-data 15  0 252M 21604 12768 S 0.0 1.0 0:15.75 /usr/sbin/apache2 -k start
    9264 www-data 16  0 252M 21520 12692 S 0.0 1.0 0:16.55 /usr/sbin/apache2 -k start
    9267 www-data 15  0 252M 21520 12688 S 0.0 1.0 0:18.10 /usr/sbin/apache2 -k start
    9268 www-data 16  0 252M 21520 12688 S 0.0 1.0 0:16.89 /usr/sbin/apache2 -k start
    9269 www-data 15  0 252M 21520 12688 S 0.0 1.0 0:16.51 /usr/sbin/apache2 -k start
    9273 www-data 16  0 252M 21520 12688 S 0.0 1.0 0:17.32 /usr/sbin/apache2 -k start
    9275 www-data 16  0 252M 21520 12688 S 0.0 1.0 0:16.03 /usr/sbin/apache2 -k start

    Summary



    The following table summarizes the above results.


    [tr]
    [td][/td]
    [td]Requests per Second[/td]
    [td]Single Request (milliseconds)[/td]
    [td]Memory (Maximum, MB)[/td]
    [td]Memory (Minimum, MB)[/td]
    [/tr]
    [tr]
    [td]None[/td]
    [td]10.41[/td]
    [td]96.08[/td]
    [td]24[/td]
    [td]24[/td]
    [/tr]
    [tr]
    [td]eAccelerator[/td]
    [td]31.26[/td]
    [td]31.99[/td]
    [td]23[/td]
    [td]18[/td]
    [/tr]
    [tr]
    [td]XCache[/td]
    [td]30.28[/td]
    [td]33.02[/td]
    [td]29[/td]
    [td]19[/td]
    [/tr]
    [tr]
    [td]APC[/td]
    [td]30.45[/td]
    [td]32.84[/td]
    [td]21[/td]
    [td]21[/td]
    [/tr]


    Conclusions



    From the above results, one can come to the following conclusions:

    All op-code caches provide a noticable improvement for Magento over a default PHP installation.
    The speed gain is about 3X.
    eAccelerator is marginally better than the XCache or APC both in terms of speed and memory utilization.
    Installation of each op-code cache is different: one has a Debian package, the other is installed from source and the third is via PECL.
    The configuration for each is also different. Some work well with a default install, others require more tweaking.


    Update:



    We have noticed that in production, Xcache suffers from the same instability that eAccelerator exhibits: segmentation fault after a day or so. This happened with Xcache 1.2.1-3 which ships with Ubuntu 8.04.1, and PHP 5.2.4. The latest stable release is 1.2.2 from Xcache's web site. So, try compiling that from source, or install the logwatcher script if a minute of downtime is acceptable.

    View more threads in the same category:


  2. #2
    New member Ritvars Radvilavičs's Avatar
    Join Date
    Oct 2014
    Location
    Leicester, UK
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Which Debian version are you working with and would suggest?
    Which one is better and why?
    Which of these Debian configurations can work with Nginx + Percona XtraDB + Amazon AWS + Redis + Varnish + PHP-FPM ?

    1.Debian Wheezy 64 bit minimal system, only SSH is installed.

    2. Debian Squeeze 64 bit + Confixx 3.3.X on Debian Squeeze 64bit base with pre-installed Parallels Confixx 3.3.9. (excl. Royalty!)

    3. Debian Wheezy 64 bit Debian Wheezy + Froxlor on 64bit base is installed and pre-configured Froxlor.

    4. Debian Wheezy 64 bit + i-MSCP Debian Wheezy 64bit on base with pre-installed i-MSCP Control Panel

    5. Debian Squeeze 64 bit + ispCP on Debian Squeeze 64bit base with pre-installed ispCP 1.1.0 Beta first

    6. Debian Wheezy 64 bit + Plesk 12.x Debian Wheezy 64bit on base with pre-installed Parallels Plesk 12 (excl. Royalty!)

    7. Debian Wheezy + OpenVPN Access Server Debian Wheezy 64bit on base with pre-installed OpenVPN Access Server (incl. 2 user license)

    8. Debian Squeeze 64 bit + ownCloud Debian Squeeze on 64bit base is installed and preconfigured ownCloud.

    9. Debian Squeeze 64 bit minimal system, only SSH is installed.

    Appreciate professional and experienced replies, thank you!

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •