BuyVM OpenVZ VPS Review: Centmin Nginx installed
First time testing out a OpenVZ based VPS server from BuyVM.net. So I posted a review over at LowEndTalk.com testing out my modified Centmin v1.2.1-eva2000.13 beta release Menu mode installer script on a BuyVM.net 512MB/1GB Burstable OpenVZ VPS server.
Interesting to learn about how OpenVZ VPS handles memory differently from other virtualization platforms and how default operating system stack sizes can cause very higher memory usage for multi-threaded services such memcached, MySQL and rsyslog. Seems the same excessive memory usage occurs with Varnish Cache too on OpenVZ based VPS servers.
About 12 days ago I nabbed 2x BuyVM OpenVZ VPS solely for live testing of my modified Centmin install script which is currently undergoing beta testing and revamp to support Menu mode installation http://vbtechsupport.com/920/. Both BuyVM OpenVZ are 512MB/1GB Burstable/50GB based VPS running CentOS 6.0 32bit.
VPS are on Intel Xeon E5520 Nehalem based processors with 2 cpu cores.
processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.978 cache size : 4096 KB physical id : 0 siblings : 6 core id : 0 cpu cores : 6 apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm bogomips : 4533.95 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Intel(R) Xeon(R) CPU L5520 @ 2.27GHz stepping : 5 cpu MHz : 2266.978 cache size : 4096 KB physical id : 0 siblings : 6 core id : 1 cpu cores : 6 apicid : 1 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc pni ssse3 cx16 sse4_1 sse4_2 popcnt lahf_lm bogomips : 4532.15 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: |
Out of the box virgin memory/disk usage and top stats:
total used free shared buffers cached Mem: 1024 51 972 0 0 0 -/+ buffers/cache: 51 972 Swap: 0 0 0 Filesystem Size Used Avail Use% Mounted on /dev/simfs 50G 566M 50G 2% / none 512M 4.0K 512M 1% /dev top - 14:22:11 up 1:05, 1 user, load average: 0.00, 0.00, 0.00 Tasks: 16 total, 1 running, 15 sleeping, 0 stopped, 0 zombie Cpu(s): 0.1%us, 0.2%sy, 0.0%ni, 99.7%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1048576k total, 53324k used, 995252k free, 0k buffers Swap: 0k total, 0k used, 0k free, 0k cached |
Disk tests
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync && rm -rf test 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 22.4788 s, 47.8 MB/s |
dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync && rm -rf test 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 7.07835 s, 152 MB/s |
My ioping.sh Disk Latency test which tests random disk speed rather than sequential disk speed http://vbtechsupport.com/1239/
----------------------------------------- ioping.sh 0.9.8 - http://vbtechsupport.com by George Liu (eva2000) ----------------------------------------- ioping.sh 0.9.8 MENU ----------------------------------------- 1. Install ioping 2. Re-install ioping 3. Run ioping default tests 4. Run ioping custom tests 5. Exit ----------------------------------------- Enter option [ 1 - 5 ] 3 ----------------------------------------- Virtuzzo OR OpenVZ Virtualisation detected *************************************************** ioping code.google.com/p/ioping/ ioping.sh 0.9.8 shell wrapper script by George Liu (eva2000) http://vbtechsupport.com *************************************************** Virtuzzo or OpenVZ Virtualisation detected ********************************** dd (sequential disk speed test)... ********************************** dd if=/dev/zero of=testfilex bs=64k count=16k conv=fdatasync 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 8.87793 s, 121 MB/s ************************ starting ioping tests... *************************************************** ioping disk I/O test (default 1MB working set) *************************************************** disk I/O: / --- / (simfs /dev/simfs) ioping statistics --- 5 requests completed in 4032.8 ms, 2648 iops, 10.3 mb/s min/avg/max/mdev = 0.3/0.4/0.4/0.0 ms ********************************************** seek rate test (default 1MB working set) ********************************************** seek rate: / --- / (simfs /dev/simfs) ioping statistics --- 370 requests completed in 3004.5 ms, 1475 iops, 5.8 mb/s min/avg/max/mdev = 0.3/0.7/16.4/1.2 ms ********************************************** sequential test (default 1MB working set) ********************************************** ----------------------- sequential: / --- / (simfs /dev/simfs) ioping statistics --- 104 requests completed in 3018.0 ms, 45 iops, 11.2 mb/s min/avg/max/mdev = 15.3/22.2/183.5/17.3 ms ----------------------- sequential cached I/O: / --- / (simfs /dev/simfs) ioping statistics --- 376 requests completed in 3005.6 ms, 4557 iops, 1139.3 mb/s min/avg/max/mdev = 0.1/0.2/13.4/0.8 ms |
Network speed
wget -O /dev/null http://cachefly.cachefly.net/100mb.test && rm -rf 100mb.test --2011-09-13 22:33:41-- http://cachefly.cachefly.net/100mb.test Resolving cachefly.cachefly.net... 205.234.175.175 Connecting to cachefly.cachefly.net|205.234.175.175|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 104857600 (100M) [application/octet-stream] Saving to: “/dev/null” 100%[=============================================================================>] 104,857,600 25.0M/s in 4.2s 2011-09-13 22:33:46 (24.0 MB/s) - “/dev/null” saved [104857600/104857600] |
2nd run
wget -O /dev/null http://cachefly.cachefly.net/100mb.test --2011-09-14 02:21:31-- http://cachefly.cachefly.net/100mb.test Resolving cachefly.cachefly.net... 205.234.175.175 Connecting to cachefly.cachefly.net|205.234.175.175|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 104857600 (100M) [application/octet-stream] Saving to: “/dev/null†100%[=============================================================================>] 104,857,600 31.5M/s in 3.6s 2011-09-14 02:21:35 (28.1 MB/s) - “/dev/null†saved [104857600/104857600] |
Apache v2.2.15 is running by default without any MySQL server installed:
httpd -V Server version: Apache/2.2.15 (Unix) Server built: Jul 7 2011 11:27:40 Servers Module Magic Number: 20051115:24 Server loaded: APR 1.3.9, APR-Util 1.3.9 Compiled using: APR 1.3.9, APR-Util 1.3.9 Architecture: 32-bit Server MPM: Prefork threaded: no forked: yes (variable process count) Server compiled with.... -D APACHE_MPM_DIR="server/mpm/prefork" -D APR_HAS_SENDFILE -D APR_HAS_MMAP -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled) -D APR_USE_SYSVSEM_SERIALIZE -D APR_USE_PTHREAD_SERIALIZE -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT -D APR_HAS_OTHER_CHILD -D AP_HAVE_RELIABLE_PIPED_LOGS -D DYNAMIC_MODULE_LIMIT=128 -D HTTPD_ROOT="/etc/httpd" -D SUEXEC_BIN="/usr/sbin/suexec" -D DEFAULT_PIDLOG="run/httpd.pid" -D DEFAULT_SCOREBOARD="logs/apache_runtime_status" -D DEFAULT_LOCKFILE="logs/accept.lock" -D DEFAULT_ERRORLOG="logs/error_log" -D AP_TYPES_CONFIG_FILE="conf/mime.types" -D SERVER_CONFIG_FILE="conf/httpd.conf" |
Before I could run my custom shell scripts, I had to at least install a few items that were missing on virgin VPS.
yum -y install gcc gcc-c++ file bc perl-TermReadKey perl-Time-HiRes perl-DBD-MySQL nano elinks |
Removing Apache v2.2.15 and install Centmin v1.2.1-eva2000.13
I ran modded Centmin install script which by default installed the following items:
- Nginx v1.0.6
- PHP 5.3.8 php-fpm custom source install
- MariaDB 5.2.7 default MySQL server – uses more memory but faster than MySQL community versions. Benchmarks
- Memcached 1.4.7 server – 2x 16MB instances including password protected memcache.php admin end
- Memcache v3.0.5 php extension
- Xcache v1.3.2 or APC 3.1.9 with respective admin browser interfaces – I choose APC 3.1.9
- Python v2.72
- CSF firewall
- Siege benchmark and Sproxy
Centmin v1.2.1-eva2000.13 beta Centmin menu version’s options
-------------------------------------------------------- Centmin 1.2.1-eva2000.13 - http://vbtechsupport.com/920/ Menu/Mods Author: eva2000 (vbtechsupport.com) Centmin Original Author: BTCentral (btcentral.org.uk) -------------------------------------------------------- Centmin Menu -------------------------------------------------------- 1). Centmin Install 2). Add Nginx vhost domain 3). NSD setup domain name DNS 4). Nginx Upgrade 5). PHP Upgrade 6). XCache Re-install 7). APC Cache Re-install 8). XCache Install 9). APC Cache Install 10). Memcached Server Re-install 11). MariaDB 5.2 Upgrade 12). Install ioping.sh vbtechsupport.com/1239/ 13). SELinux disable 14). Exit -------------------------------------------------------- Enter option [ 1 - 14 ] -------------------------------------------------------- |
I usually test my modded Centmin script on local virtualbox based CentOS 5.5/5.6/6.0 64bit servers. This was first time testing on OpenVZ VPS server and was surprised to see alot higher memory usage right after installation, particularly for Memcached servers – 2x 16MB instances took nearly 54MB each. Reading up I found it’s due to OpenVZ’s memory usage and default system stack sizes.
From http://code.google.com/p/memcached/issues/detail?id=186
On OpenVZ (operating-system level virtualization for Linux), there is no differentiation between virtual and resident memory, so all the virtual memory is counted as RAM. The default installation of Memcached therefore uses up 50MB of RAM (4+1 threads x 10MB) when starting up on an OpenVZ virtual private server.
I can confirm that this was the case on my new test buyvm OpenVZ 512MB/1GB burstable CentOS 6.0 32bit VPS. This CentOS 6.0 32bit based OpenVZ VPS has a default system stack size at 10240KB.
I wrote a quick servicemem.sh bash shell script to gather all info and tests for the below output where the script outputs base memory usage, then it stops/starts each key service and takes a free -m output after each key service has stopped/started.
As you can see on this fresh virgin Centmin v1.2.1-eva2000.13 beta Nginx/PHP-FPM installed, this OpenVZ VPS shows the following amount of memory used by each of the listed services
- with default system stack size at 10240KB
- with tweaked service init.d stack sizes
- with tweaked service init.d stack size + lighter MariaDB mysql my.cnf and apc.ini settings. Note MariaDB has higher mem usage than MySQL community versions as it has a new Aria storage engine which is needed to be running as internal on-disk tables are now in Aria format instead of MyISAM which helps speed up MySQL query performance where GROUP BY and DISTINCT occur. Read http://kb.askmonty.org/en/aria-storage-engine. I lowered the Aria log file size as well as Aria Pagecache and Sort buffer sizes from defaults which may lead to less performance.
1) Default system stack size
base mem usage
total used free shared buffers cached
Mem: 1024 328 695 0 0 0
-/+ buffers/cache: 328 695
Swap: 0 0 0 |
- rsyslod = 29MB
- lfd = 10MB
- php-fpm = 44MB
- nginx = 30MB
- mysql = 91MB
- memcached = 108MB
2) After tweaked system stack size for specific service init.d scripts
base mem usage
total used free shared buffers cached
Mem: 1024 200 823 0 0 0
-/+ buffers/cache: 200 823
Swap: 0 0 0 |
- rsyslod = 3MB
- lfd = 9MB
- php-fpm = 42MB
- nginx = 30MB
- mysql = 88MB
- memcached = 6MB
using the following stack sizes
################################## # nano -w /etc/init.d/memcached # ulimit -s 256 # service memcached restart ################################## # nano -w /etc/init.d/nginx # ulimit -s 128 # service nginx restart ################################## # 256 for 64bit & 192 for 32bit # nano -w /etc/init.d/mysql # service mysql restart ################################## # nano -w /etc/init.d/php-fpm # ulimit -s 512 # service php-fpm restart ################################## # nano -w /etc/init.d/rsyslog # ulimit -s 512 # service rsyslog restart ################################## # nano -w /etc/init.d/lfd # ulimit -s 2048 # service lfd restart ################################## |
3) Tuned MariaDB my.cnf and APC php.ini memory allocation
base mem usage
total used free shared buffers cached
Mem: 1024 114 909 0 0 0
-/+ buffers/cache: 114 909
Swap: 0 0 0 |
- rsyslod = 3MB
- lfd = 9MB
- php-fpm = 24MB
- nginx = 14MB
- mysql = 39MB
- memcached = 5MB
Geekbench v2.2.0 = 3,655 http://browse.geekbench.ca/geekbench2/view/481627
vs
Geekbench v2.2.1 beta = 3,045 http://browse.geekbench.ca/geekbench2/view/481628
Here’s a static html file siegebench.sh test run from local BuyVM VPS vs remote Wiredtree VPS to target server (BuyVM VPS).
local test = running siegebench.sh on same server as target url’s test BuyVM VPS server – buyvm OpenVZ 512MB/1GB Burstable VPS (2 core Intel Xeon E5520)
vs
remote = running siegebench.sh on remote server testing target url’s test BuyVM VPS server – wiredtree Virtouzzo 1.5GB WHM/Cpanel VPS (shared 8 core Dual Intel Xeon E5405)
local
Thu Sep 15 12:19:40 MSD 2011 Log files saved at /home/siegebenchlogs Sep 15 12:19 947 siegehtml_benchmark_150911-1218.txt ------------------------------------------------- contents of /home/siegebenchlogs/siegehtml_benchmark_150911-1218.txt ------------------------------------------------- date | trans | elapse time | data trans (MB) | response time | trans/sec | throughput (MB/s) | concurrency | success trans | failed trans ------------------------------------------------- **** siege_concurrentusers_50 **** 2011-09-15 12:18:22, 2500, 1.72, 0, 0.02, 1453.49, 0.00, 32.32, 2500, 0 **** siege_concurrentusers_100 **** 2011-09-15 12:18:32, 5000, 3.26, 0, 0.04, 1533.74, 0.00, 62.81, 5000, 0 **** siege_concurrentusers_150 **** 2011-09-15 12:18:43, 7500, 5.42, 0, 0.08, 1383.76, 0.00, 107.63, 7500, 0 **** siege_concurrentusers_200 **** 2011-09-15 12:18:56, 10000, 6.62, 1, 0.08, 1510.57, 0.15, 125.97, 10000, 0 **** siege_concurrentusers_250 **** 2011-09-15 12:19:13, 12500, 11.20, 1, 0.15, 1116.07, 0.09, 172.27, 12500, 0 **** siege_concurrentusers_300 **** 2011-09-15 12:19:34, 15000, 14.71, 1, 0.17, 1019.71, 0.07, 178.32, 15000, 0 ------------------------------------------------- |
remote
Thu Sep 15 03:25:46 CDT 2011 Log files saved at /home/siegebenchlogs Sep 15 03:25 947 siegehtml_benchmark_150911-0324.txt ------------------------------------------------- contents of /home/siegebenchlogs/siegehtml_benchmark_150911-0324.txt ------------------------------------------------- date | trans | elapse time | data trans (MB) | response time | trans/sec | throughput (MB/s) | concurrency | success trans | failed trans ------------------------------------------------- **** siege_concurrentusers_50 **** 2011-09-15 03:24:14, 2500, 5.86, 0, 0.12, 426.62, 0.00, 49.08, 2500, 0 **** siege_concurrentusers_100 **** 2011-09-15 03:24:29, 5000, 8.77, 0, 0.12, 570.13, 0.00, 65.68, 5000, 0 **** siege_concurrentusers_150 **** 2011-09-15 03:24:45, 7500, 9.19, 0, 0.12, 816.10, 0.00, 95.34, 7500, 0 **** siege_concurrentusers_200 **** 2011-09-15 03:24:57, 10000, 6.75, 1, 0.12, 1481.48, 0.15, 175.56, 10000, 0 **** siege_concurrentusers_250 **** 2011-09-15 03:25:18, 12500, 14.90, 1, 0.20, 838.93, 0.07, 169.21, 12500, 0 **** siege_concurrentusers_300 **** 2011-09-15 03:25:40, 15000, 15.95, 1, 0.22, 940.44, 0.06, 207.30, 15000, 0 ------------------------------------------------- |
