Saturday, October 29, 2011

Drizzle, NGINX, PHP and PHP-APC


This is a guide/howto on setting up Drizzle database, Nginx webserver, PHP (CGI) and PHP-apc (bytecode cache) on Debian Linux 6 (codename Squeeze).

NOTE: If the guide is too messy to read here, head over to this Google Document for better formatting.


Apache and MySQL are quite heavy and MySQL has been becoming worse thanks to the idiots in Oracle. NGINX has been getting a lot of recommendations and it’s proving itself to be quite stable and faster than any existing webserver in the arena.

Drizzle DB is the child of the co-founder of MySQL after he left MySQL when Sun bought it. Its main focus is web applications, and easy replication of databases.

From preliminary tests on a non-optimized virtual machine, I was able to reach 800 requests/sec for read/write operations and 2400 requests for read-only operations, hitting my URL shortening web application on NGINX and Drizzle. My VM had 1 core and 512MB RAM, but only 69MB RAM was used during the tests!

Stress Test

Hardware: Lenovo laptop with an Intel(R) Core(TM)2 Duo CPU P8600 @ 2.40GHz, 6GB RAM, running Debian6 32-bit.
Software: VMware Workstation v7. The VM has 512MB RAM and 1 core allocated, running Debian6 32-bit (business card installation). VMware tools were not installed during the stress tests.

I had apc statistics enabled and access logging enabled in NGINX at first, but turning them off reduced load times from 126ms to 16ms for read/write operations and from 40ms to 4ms for read-only requests.

Read/write test: ab -c13 -n 10000
Requests per second:    789.07 [#/sec] (mean)
Time per request:       16.475 [ms] (mean)
Time per request:       1.267 [ms] (mean, across all concurrent requests)
CPU utilization: 25% Drizzle, 19% NGINX, 2.3% * 15 PHP CGI processes.

Read-only test: ab -c13 -n 10000
Requests per second:    2633.09 [#/sec] (mean)
Time per request:       4.937 [ms] (mean)
Time per request:       0.380 [ms] (mean, across all concurrent requests)
CPU utilization: 49% NGINX, 3.3% * 15 PHP CGI. Drizzle wasn’t showing in “top.”

The read-only test involved engaging a REWRITE rule from NGINX, which puts a tad bit more processing on its shoulders.

The numbers above are very specific to my application, but the numbers can be much better if I install VMware tools to provide VM optimizations, so don’t let the 49% CPU usage put you off.

0) Installation
0.0) Drizzle
Note: Drizzle relies on upstart, which means sysvinit will be removed.

First, add the PPA to /etc/apt/sources.lst using your favorite editor

      deb maverick main
      deb-src maverick main

      sudo apt-key adv --keyserver --recv-keys 06899068
      sudo apt-get update
      sudo apt-get install drizzle
(to continue, you’ll have to type ‘Yes, do as I say!’ excluding the single quotes)

0.1) NGINX
      sudo apt-get install nginx

0.2) PHP

      sudo apt-get install php5-cgi php5-mysql php-apc

The PHP extension for drizzle is available as a PECL but it’s not kept up to date & since we’re gonna compile either way, we’ll have to install some dev packages & a compiler. Those can (should) be removed after the compilation of the extension, if you’re doing this on a production box.
      apt-get install php5-dev libdrizzle-dev make

This is to fix a bug in the configure script looking in the wrong place
      ln -s /usr/include/libdrizzle-1.0/libdrizzle /usr/include/libdrizzle

Grab the latest stable Drizzle PHP extension from here:
      tar -zxf drizzle-php-ext-0.5.tar.gz
      cd drizzle-php-ext-0.5
      make -j2
      make install

Let’s clean up after the mess
      make clean
      apt-get remove php5-dev libdrizzle-dev make
      apt-get autoremove
      rm -I /var/cache/apt/archives/*

In my case, the modules were copied to: /usr/lib/php5/20090626+lfs/  and the module is ready to be included in php.ini later.


1) Configuration
1.0) NGINX

The configuration of nginx is not a standard process. Each website may have its own specific setup & needs that one would have to tweak the settings around to fit one’s needs. Make sure you refer to the referenced links to see all available options and things to avoid doing.

I’ll include my own configuration changes here and not the full configuration file.

Modify the file /etc/nginx/nginx.conf
   worker_processes 3;

   error_log /var/log/nginx/error.log;
   pid /var/run/;

   events {
       worker_connections 1024;
       multi_accept on;

   http {
       include /etc/nginx/mime.types;
       access_log off;
       # if you want to have an access log, comment the line above and uncomment the following one
       # access_log /var/log/nginx/access.log;

       sendfile on;
       tcp_nopush on;

Modify the file /etc/nginx/sites-enabled/default
   index index.php index.html index.htm;

   server {
       # this block redirects all connections to to 
       # this is handy for cache configurations like Varnish & for statistics
       # if you prefer to view your site as, swap server names
       listen 80;
       rewrite ^ $scheme://$request_uri redirect;

   server {
       listen 80;
       #listen [::]:80 default ipv6only=on;

       root /var/www/;
       #access_log /var/log/nginx/localhost.access.log;

       location / {
           try_files $uri $uri/ /index.php;

       location ~ \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
           #I don’t want it to search for directories so I removed $uri/
           try_files $uri /index.php;
           #enable this if you enabled access logging previously
           #access_log off;
           expires 30d; #useful for caches

       location ~ \.php$ {
           try_files $uri /index.php;
           include fastcgi_params;
           # unix sockets are faster & better than binding to a port
           fastcgi_pass unix:/tmp/php.socket;
   # remember to include other settings from the original config file

Modify the file /etc/nginx/fastcgi_params
   fastcgi_param  SERVER_SOFTWARE    nginx;
   fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
   fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;

Create a new file /etc/init.d/php-cgi
PHP_CGI_NAME=`basename $PHP_CGI`
start() {
      echo -n "Starting PHP FastCGI: "
      start-stop-daemon --quiet --start --background --chuid "$USER" --exec /usr/bin/env -- $PHP_CGI_ARGS
      echo "$PHP_CGI_NAME."
stop() {
      echo -n "Stopping PHP FastCGI: "
      killall -q -w -u $USER $PHP_CGI
      echo "$PHP_CGI_NAME."
case "$1" in
      echo "Usage: php-fastcgi {start|stop|restart}"
      exit 1
exit $RETVAL

Run the commands:
      chmod +x /etc/init.d/php-cgi
      update-rc.d php-cgi defaults
You are now able to start the php cgi service: service php-cgi start


1.1) PHP
These are minor modifications and you should take an overall look at the config file to see if you’d like to make any other changes.

Modify the file /etc/php5/cgi/php.ini
      ; this prevents PHP from executing scripts uploaded by users into image directories
      ; required for security reasons on CGI deployments
      expose_php = Off
      ; original value is 128M which in my opinion is too much
      memory_limit = 64M
      ; before the File Uploads section, add this
      ; apc settings
      apc.shm_size = 32M
      apc.stat = 0
      ; before the Module Settings section, add this


1.2) Drizzle
      mkdir -p /home/drizzle/db/.temporary
      cp -Rv /var/lib/drizzle/* /home/drizzle/db/
      chmod -R 750 /home/drizzle
      chown -R drizzle:drizzle /home/drizzle
      mv /etc/init/drizzle.conf /etc/init/drizzle.conf.orig

Make a new file: /etc/init/drizzle.conf
      # Drizzle Service

      description     "Drizzle Server"
      author          "MBH "

      start on runlevel [2345]

      stop on runlevel [016]

      expect fork
          DRIZ_VARS="--user drizzle --datadir /home/drizzle/db --drizzle-protocol.port 3306"
          start-stop-daemon --quiet --start --pidfile /home/drizzle/db/ --chuid drizzle --group drizzle --startas /usr/sbin/drizzled -- $DRIZ_VARS
      end script

  • Drizzle doesn’t support unix sockets to make sure applications connect to rather than localhost, if the DB is local.
  • The drizzle username created in Debian doesn’t have a shell nor a home directory, and it should remain this way.
You can now start poking with Drizzle and make your own applications. Here’s a link for a quick usage of Drizzle and PHP:


No comments: