I recently decided to indulge my inner geek by doing a hardware refresh on my home ZFS file server. The system had served me well since moving to ZFS from my previous system but there was room for improvement.

Specifications

Total storage raw: 10TB

Total storage usable: 5TB

• Case
  • Norco RPC-4220 - 20 hot-swap bay, 4RU chassis
• PSU
  • Corsair HX 750W
• Motherboard
  • Supermicro X8STE - single socket 1366; 2x Intel 82574L GigE NIC; Matrox G200 GPU
• CPU
  • Intel Xeon E5620 Westmere 2.4GHz Quad Core
• RAM
  • 24GB (6x4GB) Kingston 240-pin DDR SDRAM ECC Unbuffered
• Controller Cards
  • Intel SASWT4I (LSI SAS1064E chipset) PCIe x4 4-port SAS HBA
  • LSI 9201-16i 6GB/s PCIe 2.0 x8 16-port SAS HBA
• Hard Drives
  • SanDisk 240GB SSD (SDSSDX240GG25) (boot drive)
  • 10x mixed Seagate 1TB (ST1000NM0011, ST1000NM0033) and 4TB (ST4000NM0035) Constellation ES SATA drives (zfs data pool)
• Optical Drives
  • Samsung slimline DVD-RW
• Operating System
  • FreeBSD

Description

The system is used for media (pictures, music, movies), software, backups, and personal document storage. All this data is made available via CIFS and NFS shares.

My file server is full and I have no options for expanding it. The server is a white box system running FreeBSD with a hardware RAID card and 400GB of RAID-5 storage. The hardware is old, the hard drives are old and I can't expand it. It's time for something new.

The goals of this project was to build a low-power, small form factor machine that runs OpenBSD and acts as a firewall/router in a home network or small business setting. This page walks through the hardware I chose and the process I use to get OpenBSD running on the CF card.

Table of Contents

• Hardware
• Operating System
• System Operation

Hardware

The design has gone through two generations of hardware now.

This guide will explain how to setup a site-to-site IPsec tunnel (i.e., tunnel mode IPsec) between two OpenBSD gateways. Throughout this document there are example configs shown, some of which contain secret key data. DO NOT use these example keys! Create your own (as shown) and keep them private.

The Tools

OpenBSD ships with all the tools needed to begin using IPsec. OpenBSD does not require a kernel recompile, software installtion, 3rd-party modules or anything else to get IPsec up and running.

Olive refers to a regular PC that's running Juniper Networks' JUNOS software. Juniper developed Olive early on so they could perform testing of JUNOS during development. These days Olive is deprecated in favor of cheap, low-end M and J-series routers.

CARP is the Common Address Redundancy Protocol. It's a secure, free alternative to the Virtual Router Redundancy Protocol and the Hot Standby Router Protocol. CARP was created and is maintained by the OpenBSD project.

The notes here apply to OpenBSD 5.0 and higher.

Protocol Information

Virtual MAC Address

The virtual MAC is in the format 00-00-5e-00-01-XX where the last octet is filled in by the CARP vhid.

IP Protocol

CARP uses IP protocol number 112 (0x70).

Multicast Advertisements

CARP advertisements are multicast to the 224.0.0.18 or FF02::12 multicast groups when using IPv4 and IPv6, respectively.

TTL/Hop Limit

CARP packets are always sent with a TTL/HLIM of 255 so that CARP packets that have crossed a subnet boundary (i.e., have been passed on by a router) can be recognized and dropped.

Timers

The host that advertises the most frequently will become the leader for the CARP group. The timer values configured on each host are sent as part of the CARP advertisements so that all hosts can make an accurate decision as to which host should be the leader.

OpenBGPD is a free, open-source implementation of the Border Gateway Protocol Version 4. It was created and is maintained by the OpenBSD project.

The notes here apply to OpenBGPD as found in OpenBSD 4.0 and higher.

Path Selection Process

OpenBGPD will only ever install one route in the route table for a particular destination network (prefix). If OpenBGPD receives information about that prefix from more than one peer, a decision must be made on which one to use. The prefixes received will be evaluated against each other if the follow criteria matches:

The following SNMP MIBs and the accompanying code that extend the Net-SNMP daemon allow administrators to query information from various OpenBSD subsystems. Currently, stats can be queried from:

• Packet Filter
• The kernel sensors framework
• Common Address Redundancy Protocol (CARP)

NetPacket provides a base class for a cluster of modules related to decoding and encoding of network protocol packets. Each NetPacket descendant module knows how to encode and decode packets for the network protocol it implements. Protocols that NetPacket can encode/decode include IPv4, TCP, UDP, ICMP, Ethernet, and ARP.

I've written three additional modules for NetPacket that allow the encoding/decoding of IPv6, ICMPv6, and OpenBSD's Packet Filter binary log files. I've also made numerous changes to existing modules, including fixing spelling mistakes, bug fixes, and documentation enhancements.