Learn more about the inner workings of this website’s server and how to setup an LNMP stack.
Preamble
This is not a verbose instruction set for beginners. I also did not write portions of the guide and may refer you to external websites. All resources are accurate and tested on a fresh Ubuntu 8.10 (Intrepid) Slice as of the time of this writing. Following these steps will result in a skeleton server similar to what I use. In fact, this was part of my internal documentation which I decided to make public.
I gloss over some topics such as configuration files and tuning because it is not my intention to write a comprehensive guide. This is simply a quick check sheet that I use when rolling out a new server, but it might be useful to you.
General procedure
- Initial prep and setup
- Update using Aptitude
- Create a non-privileged user
- SSH public key authentication
- Core software
- MySQL
- PHP with APC
- Nginx
- Email Services
- Security
- Disable root login
- Firewall
- Automatic security checks
- Fine tooth comb run through with penetration testing
- Monitoring
- Extra software
- logrotate
- htop
- traceroute
Initial Prep and Configuration
Update and install build tools.
1 2 3 | sudo aptitude update sudo aptitude safe-upgrade sudo aptitude install build-essential |
Create a non-privileged user
You should already know how to do this (hint: adduser jon).
SSH public key authentication
Let’s make logging into the server easier by setting up public key authentication. See How To: SSH Public Key Authentication.
Core Software
Install MySQL5 and PHP5 with APC.
1 | sudo aptitude install mysql-server php5-cgi php5-cli php5-mysql php5-mysqli php5-curl php-apc |
Setup PHP and fastcgi spawning
Create the following script in /etc/default/php-fastcgi
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | START=yes # Which user runs PHP? (default: www-data) EXEC_AS_USER=www-data # Host and TCP port for FASTCGI-Listener (default: localhost:9000) FCGI_HOST=localhost FCGI_PORT=9000 # Environment variables, which are processed by PHP PHP_FCGI_CHILDREN=4 PHP_FCGI_MAX_REQUESTS=500 |
Create the init script so that the PHP processes can be started and stopped (/etc/init.d/php-fastcgi).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | #! /bin/sh ### BEGIN INIT INFO # Provides: php-fastcgi # Required-Start: $all # Required-Stop: $all # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start and stop php-cgi in external FASTCGI mode # Description: Start and stop php-cgi in external FASTCGI mode ### END INIT INFO # Author: Kurt Zankl < [EMAIL PROTECTED]> # Do NOT "set -e" PATH=/sbin:/usr/sbin:/bin:/usr/bin DESC="php-cgi in external FASTCGI mode" NAME=php-fastcgi DAEMON=/usr/bin/php-cgi PIDFILE=/var/run/$NAME.pid SCRIPTNAME=/etc/init.d/$NAME PHP_CONFIG_FILE=/etc/php5/cgi/php.ini # Exit if the package is not installed [ -x "$DAEMON" ] || exit 0 # Read configuration variable file if it is present [ -r /etc/default/$NAME ] && . /etc/default/$NAME # Load the VERBOSE setting and other rcS variables . /lib/init/vars.sh # Define LSB log_* functions. # Depend on lsb-base (>= 3.0-6) to ensure that this file is present. . /lib/lsb/init-functions # If the daemon is not enabled, give the user a warning and then exit, # unless we are stopping the daemon if [ "$START" != "yes" -a "$1" != "stop" ]; then log_warning_msg "To enable $NAME, edit /etc/default/$NAME and set START=yes" exit 0 fi # Process configuration export PHP_FCGI_CHILDREN PHP_FCGI_MAX_REQUESTS DAEMON_ARGS="-q -b $FCGI_HOST:$FCGI_PORT -c $PHP_CONFIG_FILE" do_start() { # Return # 0 if daemon has been started # 1 if daemon was already running # 2 if daemon could not be started start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \ || return 1 start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON \ --background --make-pidfile --chuid $EXEC_AS_USER --startas $DAEMON -- \ $DAEMON_ARGS \ || return 2 } do_stop() { # Return # 0 if daemon has been stopped # 1 if daemon was already stopped # 2 if daemon could not be stopped # other if a failure occurred start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE > /dev/null # --name $DAEMON RETVAL="$?" [ "$RETVAL" = 2 ] && return 2 # Wait for children to finish too if this is a daemon that forks # and if the daemon is only ever run from this initscript. # If the above conditions are not satisfied then add some other code # that waits for the process to drop all resources that could be # needed by services started subsequently. A last resort is to # sleep for some time. start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $DAEMON [ "$?" = 2 ] && return 2 # Many daemons donâ^?~@^?~Yt delete their pidfiles when they exit. rm -f $PIDFILE return "$RETVAL" } case "$1" in start) [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" do_start case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; stop) [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" do_stop case "$?" in 0|1) [ "$VERBOSE" != no ] && log_end_msg 0 ;; 2) [ "$VERBOSE" != no ] && log_end_msg 1 ;; esac ;; restart|force-reload) log_daemon_msg "Restarting $DESC" "$NAME" do_stop case "$?" in 0|1) do_start case "$?" in 0) log_end_msg 0 ;; 1) log_end_msg 1 ;; # Old process is still running *) log_end_msg 1 ;; # Failed to start esac ;; *) # Failed to stop log_end_msg 1 ;; esac ;; *) echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2 exit 3 ;; esac |
Now finish up
1 2 | sudo chmod +x /etc/init.d/php-fastcgi sudo /usr/sbin/update-rc.d -f php-fastcgi defaults |
Download, compile, and install Nginx.
1 2 3 4 5 6 7 8 9 10 11 12 13 | sudo aptitude install libpcre3 libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev mkdir ~/sources cd ~/sources # replace version numbers with most recent stable version wget http://sysoev.ru/nginx/nginx-0.6.35.tar.gz tar -zxvf nginx-0.6.35.tar.gz cd nginx-0.6.35/ ./configure --sbin-path=/usr/local/sbin --with-http_ssl_module --with-http_stub_status_module --pid-path=/var/run/nginx.pid --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log make # if upgrading, stop nginx first. Zero downtime upgrades are more complex. Reference the Nginx Wiki if it's that important. make install |
Create the Nginx init script (/etc/init.d/nginx)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | #! /bin/sh ### BEGIN INIT INFO # Provides: nginx # Required-Start: $all # Required-Stop: $all # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: starts the nginx web server # Description: starts nginx using start-stop-daemon ### END INIT INFO PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin DAEMON=/usr/local/sbin/nginx NAME=nginx DESC=nginx test -x $DAEMON || exit 0 # Include nginx defaults if available if [ -f /etc/default/nginx ] ; then . /etc/default/nginx fi set -e . /lib/lsb/init-functions case "$1" in start) echo -n "Starting $DESC: " start-stop-daemon --start --quiet --pidfile /var/run/$NAME.pid \ --exec $DAEMON -- $DAEMON_OPTS || true echo "$NAME." ;; stop) echo -n "Stopping $DESC: " start-stop-daemon --stop --quiet --pidfile /var/run/$NAME.pid \ --exec $DAEMON || true echo "$NAME." ;; restart|force-reload) echo -n "Restarting $DESC: " start-stop-daemon --stop --quiet --pidfile \ /var/run/$NAME.pid --exec $DAEMON || true sleep 1 start-stop-daemon --start --quiet --pidfile \ /var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS || true echo "$NAME." ;; reload) echo -n "Reloading $DESC configuration: " start-stop-daemon --stop --signal HUP --quiet --pidfile /var/run/$NAME.pid \ --exec $DAEMON || true echo "$NAME." ;; status) status_of_proc -p /var/run/$NAME.pid "$DAEMON" nginx && exit 0 || exit $? ;; *) N=/etc/init.d/$NAME echo "Usage: $N {start|stop|restart|reload|force-reload|status}" >&2 exit 1 ;; esac exit 0 |
Finish preparing the init script.
1 2 | sudo chmod +x /etc/init.d/nginx sudo /usr/sbin/update-rc.d -f nginx defaults |
Configure Nginx and your home directory structures
Refer to the following documents on creating your Nginx configuration and home directory structures:
Nginx Configuration Source Layout
Sample Nginx Configuration
Nginx Virtual Hosts – Part 1
Nginx Virtual Hosts – Part 2
In a nutshell: your config files should be in /usr/local/nginx/conf and your home directories could be laid out as follows.
1 2 3 4 | /home//public_html//log /home//public_html//backup /home//public_html//public /home//public_html//private |
There is no right or wrong way (although it’s debatable), so use what works for you.
Setup Email Services
I was previously using a Postfix + Courier setup on Ubuntu Hardy/Intrepid, however I am now using Postfix + Dovecot for better performance.
Follow my new guide on Setting Up Email Services on Ubuntu Intrepid using Exim and Dovecot. Or, you can follow my old guide on Setting Up Email Services on Ubuntu Hardy using Postfix and Courier, which also works on Ubuntu Intrepid.
Security
Disable root login
Add your non-privileged user to sudoers using visudo and then disable root login in /etc/ssh/sshd_config. For added security, change your SSH port to something unusual. Finally, make sure that you are using very strong passwords across the board.
Firewall
Setup iptables to you hearts content. There is a simple tutorial under the corresponding section header on http://articles.slicehost.com/2008/11/28/ubuntu-intrepid-setup-page-1. Additionally, you should explore adaptive solutions such as fail2ban.
Automatic security checks
Scans with chkrootkit
Scans with rkhunter
Fine tooth comb through with penetration testing
No one knows your setup better than you, so think like a hacker to find the weak spots. For example, do you let people upload files? If so, can these uploaded files be executed? Make sure that your code is safe. There are enough security vulnerabilities introduced by other peoples code.
Monitoring
Put the final touches on by adding monitoring utilities. Personally, I’m using Pingdom with custom health monitoring scripts.
Other Programs
logrotate
You need to rotate the logs periodically so that they don’t become space hogs and bring everything to a crawl.
1 | sudo aptitude install logrotate |
Nginx was the only software that we installed outside of the package manager, so that’s the only one that we need to specifically setup with logrotate. All the other logs should theoretically be taken care of.
Create the logrotate rules for nginx in /etc/logrotate.d/nginx
1 2 3 4 5 6 7 8 9 10 11 | /home/*/public_html/*/log/*.log /var/log/nginx/*.log { daily missingok rotate 7 notifempty create 640 root adm sharedscripts postrotate [ ! -f /var/run/nginx.pid ] || kill -USR1 `cat /var/run/nginx.pid` endscript } |
htop—htop is a very useful little program, but is not required.
1 | sudo aptitude install htop |
traceroute—Most useful networking tool in the world.
1 | sudo aptitude install traceroute |
unzip—Useful for those annoying files that are .zip and not .tar.gz
1 | sudo aptitude install unzip |
AWStats—Useful in processing Nginx logs.
1 | sudo aptitude install awstats |
Create the cache directory
1 | sudo mkdir /var/cache/awstats |
Now add your configurations in /etc/awstats. For example, /etc/awstats/awstats.jonsview.com.conf looks like
1 2 3 4 5 6 7 8 9 | LogFile="/home/jon/public_html/jonsview.com/log/access.log" LogFormat=1 DNSLookup=0 DirData="/var/cache/awstats/" DirCgi="/cgi-bin" DirIcons="/icon" SiteDomain="jonsview.com" AllowToUpdateStatsFromBrowser=1 AllowFullYearView=3 |
Because you’re using Nginx, the following Slicehost thread will be very beneficial on accessing your statistics: http://forum.slicehost.com/comments.php?DiscussionID=2561
You will probably want to setup automatic updating before the logs are rotated every night.
Change Log
3/27/2009 – Added PHP APC, unzip, AWStats, and fail2ban recommendation.

{ 1 comment… read it below or add one }
I like that, Jon .. I like that a lot. I’d like it if you searched for similar stuff on my blog, we could compare notes, why not ..
You’ve helped me on those log files here, especially .. most appreciated, thank you.