I took the time to try out the new FreeBSD release 8.0, so I downloaded the iso from the FreeBSD site, burned it and installed it on my home server (the server I’m referring to is in this post). After reinstalling all the ports I had on it previously (Apache, MySQL, PHP5, PHP5-extensions, Java 1.6, Openfire etc.), I decided to go one step further and try something I hadn’t done before, get email working.

DNS settings

To be able to receive email I needed a domain, now it happens that I purchased chrismcdonald.co.uk at some point last year, got it very cheap for 2 years from 123-reg. The domain wasn’t really doing anything and was just being used to forward to this blog (and still is), so I decided to get more use out of it and use a subdomain to point to my home server. I set the hostname of my FreeBSD server to server.chrismcdonald.co.uk by adding hostname=”server.chrismcdonald.co.uk” to /etc/rc.conf and added an entry in /etc/hosts so that it would resolve locally. So my hosts file now contained:

127.0.0.1               localhost server.chrismcdonald.co.uk

To get the domain to point to my home server I changed my dns records for chrismcdonald.co.uk and added a CNAME record pointing server.chrismcdonald.co.uk to my dynamic dns host chris.is-a-geek.net. I also added an MX record for the root domain which directs email to chrismcdonald.co.uk to chris.is-a-geek.net. Dynamic DNS allows users with a non-static IP (most home internet users) to use a host name to refer to your system’s ip address, which might change whenever you connect to your ISP. It works by having your home system contact the dynamic dns provider every so often to make sure the ip address mapping is kept up to date. I use dyndns.com, you can create a free account with them where you can create some free host names to use. You get to pick your own subdomain from any one of the domains they allow you to use, I chose the domain is-a-geek.net and my subdomain chris. My wireless router has built in support to work with dyndns.com and keep my host ip address updated, but if yours does not, there are some instructions also in my older post to get this working on FreeBSD.

Receiving mail

To ensure I could send and receive email past my wireless router, I added some port forwards for ports 25 (smtp), 465 (secure smtp) and 993 (secure imap), I already have existing port forwards set up for web and ssh access. Sendmail is configured by default to only allow sending of email and receiving mail only from localhost. To enable sendmail to receive mail externally I added this line to /etc/rc.conf and started sendmail again.

sendmail_enable="YES"

/etc/rc.d/sendmail stop
/etc/rc.d/sendmail start

OK so with the domain name sorted out and sendmail set to receive external mail I tried testing it out. I logged into another remote server, which was located outside of my local network on the internet. I used telnet to attempt to manually send an email. Below is the transcript, my commands are in bold, square brackets are for your server name and email.

telnet server.chrismcdonald.co.uk 25
Connected to chris.is-a-geek.net.
Escape character is '^]'.
220 server.chrismcdonald.co.uk ESMTP Sendmail 8.14.4/8.14.3; Sat, 8 May 2010
12:08:15 +0100 (BST)
HELO [remote server name]
250 server.chrismcdonald.co.uk Hello [remote server name and ip] , pleased
to meet you
MAIL FROM: [user@remoteserver]
250 2.1.0 [user@remoteserver]... Sender ok
RCPT TO: [user@server - in this case chris at chrismcdonald.co.uk]
550 5.7.1 [user@server]... Relaying denied
QUIT
221 2.0.0 server.chrismcdonald.co.uk closing connection

So it appeared that my server was not accepting email for my domain. To fix this I created the file /etc/mail/local-host-names and added the following lines, you would add your own hostnames.

chrismcdonald.co.uk
server.chrismcdonald.co.uk

Then when I tried again:

telnet server.chrismcdonald.co.uk 25
Connected to chris.is-a-geek.net.
Escape character is '^]'.
220 server.chrismcdonald.co.uk ESMTP Sendmail 8.14.4/8.14.3; Sat, 8 May 2010
12:08:15 +0100 (BST)
HELO [remote server name]
250 server.chrismcdonald.co.uk Hello [remote server name and ip], pleased
to meet you
MAIL FROM: [user@remoteserver]
250 2.1.0 [user@remoteserver]... Sender ok
RCPT TO: [user@server - in this case chris at chrismcdonald.co.uk]
550 5.7.1 [user@server]... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Subject: test email again
Just testing
.
250 2.0.0 o48BLo6R001500 Message accepted for delivery
QUIT
221 2.0.0 server.chrismcdonald.co.uk closing connection

Sure enough I had the new email in my local mailbox (checked by running mail).

IMAP access

So now I could receive mail from outside, I looked around for a few guides to get started on setting up secure IMAP and secure SMTP sending. I needed a program that would provide IMAP access to mail and came across this blog post on Dovecot on freebsddiary.org, this is a great site with loads of tutorials and info on FreeBSD. I followed the steps in the guide to install Dovecot with some minor adjustments.

cd /usr/ports/mail/dovecot
make install clean

Below are the make options I used:

Dovecot make options

You can follow those steps from freebsddiary if you are starting from scratch. I ignored the certificate stuff because I already had a self signed one created, I also chose to use PAM authentication, which uses your user account password to authenticate. The important bits which I changed from /usr/local/etc/dovecot.conf are below, change them where they occur in the file:

protocols = imaps
ssl_cert_file = /etc/ssl/certs/server.chrismcdonald.co.uk.cert
ssl_key_file = /etc/ssl/private/server.chrismcdonald.co.uk.key
ssl_key_password = ******************
mail_location = mbox:~/mail/:INBOX=/var/mail/%u
protocol imap {
listen = *:143
ssl_listen = *:993
# Login executable location.
...
}
auth default {
  # Space separated list of wanted authentication mechanisms:
  #   plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey
  #   gss-spnego
  # NOTE: See also disable_plaintext_auth setting.
  mechanisms = plain login
...
}

Then add the following to /etc/rc.conf and start dovecot.

dovecot_enable="YES"

/usr/local/etc/rc.d/dovecot start

Now I access my mail using imap securely over port 993. So far the server is able to receive mail, send mail (only from localhost) and has secure imap access to read mail, if this is suitable for what you need you can stop here. If you want to be able to send mail from another machine using your secure smtp read on.

SMTP sending

I found a couple more guides on having secure smtp authentication here and here. I did not follow those steps but borrowed some from both guides. The common setup appears to use cyrus sasl (Simple Authentication and Security Layer) for the authentication and to configure sendmail to use it. SASL SASL support is not build into sendmail by default, the guides said to recompile sendmail from its source /usr/src/usr.sbin/sendmail which I couldn’t find, I guess these things are all done using the ports system now, so here’s what I installed.

cd /usr/ports/security/cyrus-sasl2
make install clean
cd /usr/ports/security/cyrus-sasl2-saslauthd
make install clean
cd /usr/ports/mail/sendmail-sasl
make install clean

Then I needed to enable the sasl daemon by editing /etc/rc.conf and adding this line and starting the sasl daemon.

saslauthd_enable="YES"

/usr/local/etc/rc.d/saslauthd start

Finally sendmail needed to be configured to use secure smtp authentication, I also copied my certificate files that I generated a while back into /etc/mail/certs. I copied the default freebsd.mc and freebsd.submit.mc files located in /etc/mail to server.chrismcdonald.co.uk.mc and server.chrismcdonald.co.uk.submit.mc and edited server.chrismcdonald.co.uk.mc. Then following bits from both guides I added the following:

dnl set SASL options
TRUST_AUTH_MECH(`GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
define(`confAUTH_MECHANISMS', `GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN')dnl
dnl Offer SMTP AUTH only after encryption (STARTTLS) has been negotiated
define(`confAUTH_OPTIONS',`p,y')dnl

define(`CERT_DIR', `/etc/mail/certs')dnl
define(`confCACERT_PATH', `CERT_DIR')dnl
define(`confCACERT', `CERT_DIR/server.chrismcdonald.co.uk.cert')dnl
define(`confSERVER_CERT', `CERT_DIR/server.chrismcdonald.co.uk.cert')dnl
define(`confSERVER_KEY', `CERT_DIR/server.chrismcdonald.co.uk.key')dnl
define(`confCLIENT_CERT', `CERT_DIR/server.chrismcdonald.co.uk.cert')dnl
define(`confCLIENT_KEY', `CERT_DIR/server.chrismcdonald.co.uk.key')dnl
DAEMON_OPTIONS(`Port=smtps, Name=TLSMTA, M=s')dnl

The lines starting with dnl are comments and those backticks ` are not typos, the opening quote is a backtick and the closing one a normal apostrophe. Finally after running the following in /etc/mail I had secure smtp authentication working:

make
make install
make restart

Well, that’s it for another post, sorry it was such a long one. I am buying a couple of real servers on eBay that wer e going pretty cheap, the kind they use in datacentres. I will probably have a go at getting internal dns working, so that I won’t be restricted to a single server working at home. If it works I will do another post on it.

Installing Apache

We are now going to install apache from the ports collection, remember I said the ports collection was important and that you definitely wanted to include it in the install process? The ports collection is a massive collection of free software, mostly source code which has been specifically ported to work on FreeBSD. You install a port by compiling the source code and copying the compiled binary files to the relevant places. Fortunately you don’t have to do this manually, the ports have a Makefile which will do most if not all the work for you.

Let’s start by installing the latest version of apache 2.2 which is located in /usr/ports/www/apache22, you will see there is a Makefile. You can install almost any port with a make file by just running make install clean, in some cases you can run make config to set any configuration options before actually compiling and installing, this applies here, run make config,  you will be presented with a window with some configuration options, you can change these if you want if you already know there are certain Apache requirements you want, otherwise accept the defaults, press the tab key so OK is selected and press enter. Then run make install clean, it will then go about downloading,  compiling the source code, install the files and any required dependencies. Make a cup of tea, this might take a while. Whenever you are installing software on your FreeBSD server it is always best to do it via the ports collection, firstly because you are getting the software that is most likely to work on your system. Secondly when you install a port, other ports will know about the installed software, thereby the installed port can be used by other ports as a software dependency because the system knows where those installed port programs should be. Thirdly the dependencies are all managed for you. If one port relies on some other software that is not installed, in most cases the make file will go about installing that port for you, in some cases, especially when installing an upgraded port you might run into problems you have to solve manually. For instance, when I tried upgrading apache, I ran into this problem:

Found libtool-1.5.26, but you need to upgrade to libtool>=2.2

The make process failed for apache because of this dependency that it couldn’t solve automatically, in this case I had to find out where the libtool port existed (/usr/ports/devel/libtool22) and install that first. You may run into this kind of problem along the way when administering your server but once you get used to installing ports you will find it a walk in the park,the internet is your friend if you run into any problems. The final reason why you should install from the ports collection is a combination of all three, it’s easier!

After apache has finished installing, you need to add a line in /etc/rc.conf so that apache will be started automatically when the computer first starts, as root edit rc.conf and add the following.

apache22_enable="YES"

Installing MySQL Server

Now lets install mysql server, there are a few different versions to choose from, I would recommend going with 5.1, you can try 6.0 if you want. Go to /usr/ports/databases/mysql51-server and type:

make install clean

This is going to take a very long time, longer than apache. When its finished, type rehash, this will refresh you shell path variables so that you will have access to the new binaries installed with mysql server.  After that, you can run the mysql install script which initialises mysql server. Just type mysql_install_db. To make mysql automatically start like we did for apache you need to the following to /etc/rc.conf:

mysql_enable="YES"

You will probably want to change the mysql root password, I think by default it doesn’t require a password and that is probably a bad thing. Just like in unix where the root account can do anything, so to in mysql the root account can do anything to any database on the system. To set a password use:

mysqladmin -u root password 'mypassword'

The quotes are important here. Now if you restart your machine you should have both apache and mysql running, but you don’t have to restart to start using them. In FreeBSD most software that runs as a long term process (i.e. a daemon) is stored in the rc.d directory and are avaible to start and stop. System daemons you will find in /etc/rc.d/, user installed programs such as mysql-server and apache are found in /usr/local/etc/rc.d/. These programs are actually known as rc scripts, many of them can be configured to start when the system starts up. They all usually have a common way to start and stop them, for instance you can do the following for mysql-server:

/usr/local/etc/rc.d/mysql-server start
/usr/local/etc/rc.d/mysql-server stop
/usr/local/etc/rc.d/mysql-server status

These are pretty self explanatory, the status option shows whether or not the daemon is running. These scripts start the processes making sure they run under the correct user (mysql runs under the user mysql, apache runs by default under the www user) and performing the necessary commands to start or stop the process. You can start and stop mysql server this way, but for apache there is a better command called apachectl. If you need to start apache you can run apachectl start, if you want to restart it (because you’ve made some config changes) its best to use apachectl graceful, this will keep any clients connected and do a safe restart.

Installing PHP 5

Now on to PHP and the php extensions, this port is located in /usr/ports/lang/php5. This has the same process, I’ll put all the steps in the one go:

make install clean
cd /usr/ports/lang/php5-extensions
make install clean

For the php extensions you can choose what you need, definitely enable mysql if you are planning to work with mysql and php.

Finally we have a few steps to perform on our apache config file to get php to work with it and to change which files our web server points to. As root edit the apache config file:

edit /usr/local/etc/apache22/httpd.conf

Change the document root to where ever you want, there are two places to change this. This is the default directory on the system that you want apache to serve on the web.

#
# DocumentRoot: The directory out of which you will serve your
# documents. By default, all requests are taken from this directory, but
# symbolic links and aliases may be used to point to other locations.
#
DocumentRoot "/usr/local/www/apache22/data"

#
# This should be changed to whatever you set DocumentRoot to.
#
<Directory "/usr/local/www/apache22/data">

Ok, now we need to make sure php files are recognised by apache, do a search for DirectoryIndex in the file and add index.php before index.html as below. You can change this according to which file you want apache to give preference to, if you put index.php first then apache will look first for an index.php before index.html in a folder to serve.

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
 DirectoryIndex index.php index.php3 index.html index.htm
</IfModule>

Now lets make sure that the php install process has added the php extension in apache. Make sure that this extension is in the exensions list, if not, add it.

LoadModule php5_module  libexec/apache22/libphp5.so

Nearly done, now add the two lines at the end of the file, which tell apache that php and phps files should be handled by php.

AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps

That should be it, you now have a FAMP server! Now lets make sure you can get to it from outside.

Setting up Dynamic DNS

In order to get over the problem of your broadband provider assigning you ip addresses that can change at any time you use dynamic dns. This involves installing a program which periodically updates a server with your current IP address. I used dyndns for my home server. They are very good and charge nothing. The way it works is you can sign up for a free account with them, you get to pick a subdomain and one of their many domain names. For instance I chose chris.is-a-geek.net. You can then download their client program which periodically updates their servers with your current ip address, so that anyone that types in for instance, http://chris.is-a-geek.net will end up at my current ip address. Now luckily for me, my wireless router has a section in its admin interface for configuring dynamic dns and DynDNS.org (.org or .com its the same site) is a supported option. I just enabled the Dynamic DNS option on my router and entered my DynDNS account username and password. With this up and running it meant my router was now contactable via the domain chris.is-a-geek.net. Now together with the router port forwards that I mentioned in my previous post I now had a web server that is accessible via the internet, and you can too!

If your broadband router does not have an option for dynamic dns, don’t dispair! There is a port for a dynamic dns client on FreeBSD called ddclient, to get it working follow these steps:

cd /usr/ports/dns/ddclient
make install clean
cd /usr/local/etc
cp ddclient.conf.sample ddclient.conf
edit ddclient.conf

Do a search for a line that starts with use=web, remove the comment # to activate that line, it should say the following:

use=web, web=checkip.dyndns.org/, web-skip='IP Address'

Now uncomment the section for your dynamic dns provider and enter your account login details, domain name and save.
Remember I mentioned rc scripts, well this has one too, you can enable ddclient to start when the system starts by editing rc.conf and adding the following lines:

ddclient_enable="YES"
ddclient_flags="-daemon 600"

This tells ddclient to start on system start up and to update the dynamic dns provider with the current ip address every 600 seconds. You can change this value, don’t set it too low or your dynamic dns provider might get annoyed and block you, don’t set it too long either. As ddclient has an rc script you don’t have to wait to restart for it to work, just run as root:

/usr/local/etc/rc.d/ddclient.sh start

You should now have an accessible web server! You can point existing domains you own to your home server by adding a CNAME (canonical name) record in your domain dns records. For instance I have set up a CNAME dns record for the subdomain server.opencoder.co.uk to point to chris.is-a-geek.net (if you ping both of these you will see they resolve to the same ip address). This way you can point many domains to your home server and be able to run different websites for those domains by setting up apache virtual hosts (I won’t cover this here, maybe in a later post).

That’s it for now, there are many more things you can put on your server to make it even more useful. I may do more posts in the future covering a few of them. That’s enough of this for now, I need to get more techy and start writing about programming, anyway hope you found this info useful. By they way if you are interested in learning more about freebsd I’d highly recommend getting a copy of Absolute FreeBSD 2nd Edition by Michael W. Lucas and for instructions on using FreeBSD as a server, Building a server with FreeBSD 7 by Bryan J. Hong. There are also a few good posts about FreeBSD port management and installing FreeBSD with nice screenshots at my work colleague Luis’ blog http://www.lupomontero.com. We are working closely with Luis on an interesting project called StickyWorld, which involves posting stickynotes on work in realtime in a flash based ‘room’. I will no doubt be writing about that soon, until then, have fun with your server.