packetmischief.ca
Code
Internetworking
Juniper Networks
Network Monitoring
OpenBSD
OpenBSD CF Router
Solaris ZFS Server
Contact Me

OpenBSD Compact-Flash Router

Goals

The goals of this project were simple: design and build a router out of a PC which will run OpenBSD. It must have a minimum of three physical Ethernet interfaces, each capable of 802.1q VLAN tagging. It must use a compact flash card for permanent storage. It should have a small form factor, low power consumption, and have very low noise and heat emissions.

Hardware

The original hardware was:

A Shuttle system was chosen for a few reasons:

The SV25 model was chosen specifically because it supported the VIA C3 processor.

The VIA C3 processor was chosen for its low power consumption properties and also because it's possible to run them without a cooling fan which can be very advantagious when building a low-noise system. The 933MHz model was chosen mostly because the price was right and partially because enough CPU power was needed to be able to terminate a couple VPN tunnels on the router without impacting performance. Unfortunately, the Ezra core in this CPU doesn't support VIA's PadLock Hardware Security technology which can really boost performance of crypto operations (and which is supported in OpenBSD).

The amount of RAM was dictated by the need to have a few RAM drives present so that continual and/or frequent writes to the Compact Flash card could be avoided.

The Compact Flash card was chosen for the router's permanent storage because it makes no noise, emits very little heat (if any), and has no moving parts and is therefore much less prone to failure than a hard drive. The size of the card was dictated by an evaluation of the size of a fresh OpenBSD install and by price. The card plugs into a cfdisk.1c Compact Flash to IDE adapter card which makes it look like an IDE hard drive to the operating system.

The quad port Ethernet card provides the minimum number of physical interfaces to meet the design goal. The Phobos P430/TX supports dot1q tagging and also employs the DEC 21142/3 chipset which is both well supported and robust. As an added bonus, the on-board Realtek 8139 NIC functions under OpenBSD which bumps the number of physical interfaces to five.

In the fall of 2007, the system started to randomly panic and crash. I replaced everything I could (except CPU and motherboard) but the problems continued. I had to retire the Shuttle hardware and move my flash card into some new hardware.

The New Hardware

After the Shuttle system started having problems, I moved the flash card over to an Acrosser AR-M9936 network appliance. These systems are pretty cool:

A dmesg of this system is here: dmesg.txt

Operating System

The base operating system is OpenBSD, but it requires a few changes to adapt it to the router. These are the major changes:

RAM drives are created for areas of the filesystem that are written to frequently such as /dev and /var. Since Compact Flash drives can only be written to a finite number of times, using RAM drives will extend the life of the flash drive. The following RAM drives (aka, Memory File Systems) are created: 

Filesystem     1K-blocks     Used    Avail Capacity  Mounted on
mfs:18756         375        21       336     6%     /dev
mfs:1886        19327         2     18359     0%     /tmp
mfs:28310       19327      2612     15749    14%     /var

/dev is populated on boot using MAKEDEV(8). MAKEDEV normally resides in /dev, but because /dev is an empty RAM drive, MAKEDEV has been moved to /sbin.

The /var disklabel is a tricky one to store on a RAM drive because there are files stored there that need to be available after the system reboots. As things are now, once the system starts up, /var will be an empty RAM drive. It gets populated from a "skeleton" directory /var.skel which contains the basic directory structure and files needed to get /var into a useable state. When the system restarts, any changes that were made to any files or directories on a RAM drive are lost. This impacts two directories of importance: /var/cron and /var/db/pkg. The solution is to store these directories on the actual flash drive itself. A symbolic link for each directory is created back into /etc in order to accomplish this.

root@mu:~% ls -l /var/cron /var/db/pkg
lrwxr-xr-x 1 root  wheel   9 May  4 20:14 /var/cron -> /etc/cron
lrwxr-xr-x 1 root  wheel  11 May  4 20:14 /var/db/pkg -> /etc/db/pkg

Installation

Installing the operating system is made up of two parts:

NOTE: This installation process assumes the reader is familiar with the OpenBSD install procedure. The process below is not refined or fool-proof by any means. Be careful of what you're doing.

Building the file set involves taking the stock OpenBSD file sets and making the necessary changes to them for operation on the router. Some of these changes are detailed above. Since this task can be time consuming and tedious, a build script does all of the hard work. The build script and support files are available further down.

Once the build script has finished, you'll end up with a file set named muXX.tgz where XX is the version of OpenBSD being installed. The file set is installed by booting the OpenBSD install media and following this procedure (it's assumed that the file set is placed on an FTP server):

# newfs /dev/wd0a
# mount /dev/wd0a /mnt
# newfs /dev/wd0e
# ftp -ao - -V -m ftp://server/dir/muXX.tgz | tar zxphf - -C /mnt
# cd /mnt/dev && ./MAKEDEV all
# cat /usr/mdec/boot > /mnt/boot
# /usr/mdec/installboot -v /mnt/boot /usr/mdec/biosboot wd0

Reboot the system. It will boot into a generic OpenBSD install. Network interface configuration, timezone configuration, setting the root password, etc, will all have to be done at this point.

Operation

Some notes on the router's operation:

The original Shuttle system was built in February 2004 and ran until the fall of 2007 when I moved everything to the Acrosser appliance.

Download

The build script and support files are available here: mu_build44.tar.

This framework supports build and installation of OpenBSD 4.4.

Gallery

Pictures of the router can be found here: gallery.

Copyright © 2004-2009 Joel Knight