Tag Archives: notes

Why I Use MediaWiki for Taking Notes

I was prompted to write this when I observed someone the other day who was sitting in the same training as me taking notes in a self-addressed email. No offense to people who do this, but W. T. F. How are you going to keep track of that email among the dozens/hundreds you receive every single day?

I take a lot of notes for research, certification study, and training. I use MediaWiki for almost all of these notes. Here’s why. Continue reading Why I Use MediaWiki for Taking Notes

OpenBSD OpenBGPD Notes

OpenBGPD is a free, open-source implementation of the Border Gateway Protocol Version 4. It was created and is maintained by the OpenBSD project.

The notes here apply to OpenBGPD as found in OpenBSD 4.0 and higher.

Path Selection Process

OpenBGPD will only ever install one route in the route table for a particular destination network (prefix). If OpenBGPD receives information about that prefix from more than one peer, a decision must be made on which one to use. The prefixes received will be evaluated against each other if the follow criteria matches:

  • Prefix length is the same
  • Both routes are for the same destination network
  • The NEXT_HOP is reachable

OpenBGPD uses the following process to determine the “best” route:

  1. Local Preference. Higher local preference is preferred.
  2. AS_PATH length. Shorter AS_PATH is preferred.
  3. Origin code. Lower origin code is preferred. (IGP > EGP > Incomplete)
  4. Multi Exit Discriminator. Lowest MED is preferred. Note that MEDs are only compared if the two routes where announced to us from the same AS. (The MED can be used even if the peers are in different ASes using rde med compare always)
  5. Prefer eBGP learned routes over iBGP learned routes.
  6. Weight. This is administratively set using the weight keyword in the bgpd config file. Note that by default, the weights are equal.
  7. Route age. The older route (i.e., the more stable route) is preferred. Note that this step is only performed when the rde route-age option is set to evaluate. By default it is set to ignore.
  8. BGP ID of the peer that announced the route to us. Lowest BGP ID is preferred.
  9. Lowest peer address. This is the final, tie-breaking rule. The BGP peer that has the lowest IP address wins.

If at any step in the process a route is found to be more preferred than the other, the process is aborted and that route is taken as the “best” route to the destination network.

UPDATE (Aug 2012): The bgpd path selection process is now documented in the bgpd(8) man page.

Next Hop Resolution

When looking for and validating next-hop addresses, OpenBGPD will only consider static routes and routes added by other dynamic routing protocols. BGP-learned routes and the default route are not considered. This behavior can be overridden using the nexthop qualify via configuration option.

BGP Path Attributes

The following BGP path attributes can be controlled via bgpd.conf.

Default value for routes received from iBGP peers is 100. Default value for routes recieved from eBGP peers is 0. LOCAL_PREF is never advertised to eBGP peers.
The MED is only advertised to an eBGP peer if it’s being sourced from the local router. The MED is always advertised to iBGP peers (if it’s present for a given prefix).
Communities are advertised to all peers (iBGP and eBGP).
By default the next-hop is set to whatever the peer advertises as the next-hop. The next-hop can be set to an alternate address or can be used to create null routes by using the blackhole or reject target.
Weight is a local attribute and is never redistributed to peers. It is used as tie-breaking criteria when comparing two equal routes.

Refer to the bgpd.conf(5) man page for details on how to manipulate the path attributes.


OpenBSD CARP Notes

CARP is the Common Address Redundancy Protocol. It’s a secure, free alternative to the Virtual Router Redundancy Protocol and the Hot Standby Router Protocol. CARP was created and is maintained by the OpenBSD project.

The notes here apply to OpenBSD 5.0 and higher.

Protocol Information

Virtual MAC Address
The virtual MAC is in the format 00-00-5e-00-01-XX where the last octet is filled in by the CARP vhid.
IP Protocol
CARP uses IP protocol number 112 (0x70).
Multicast Advertisements
CARP advertisements are multicast to the or FF02::12 multicast groups when using IPv4 and IPv6, respectively.
TTL/Hop Limit
CARP packets are always sent with a TTL/HLIM of 255 so that CARP packets that have crossed a subnet boundary (i.e., have been passed on by a router) can be recognized and dropped.


The host that advertises the most frequently will become the master for the CARP group. The timer values configured on each host are sent as part of the CARP advertisements so that all hosts can make an accurate decision as to which host should be master.

Advertisement Interval
This is the base interval at which CARP advertisements will be sent. The default is 1 second and is configured with the advbase keyword. In OpenBSD 5.0 onwards, a value of 0 may be specified if sub-second failover between nodes is required.
Advertisement Skew
This value is used to skew the advertisement interval of a host in order to make it more or less preferred in becoming master. A higher skew value causes a host to send CARP advertisements a fraction of a second slower than hosts with a lower value thereby making it less preferred as the master. The valid range is 0 to 254 with the default being 0. Configure skew with the advskew keyword.

When advbase is set to 0 the skew value alone is used to calculate how often advertisements are sent (the advertisement window) using this formula:

window in microseconds = advskew * 1000000 / 256

Eg: 100 * 1000000 / 256 = 390625 µs

As shown, a skew value of 100 (and interval value of 0) results in an advertisement window of 0.39 seconds.

Failover Timer
If a backup CARP host doesn’t see an advertisement from the master for 3 consecutive advertisement windows then it assumes the master is down. It will take over the CARP group and start advertising itself as the master. The number of advertisement windows to delay before assuming the master is down is hard-coded into CARP and is not tunable.

In the event that two or more hosts have the same timer values configured, the following behavior results:

  • If preempt is disabled: whichever host starts advertising first (i.e., is configured first) will become the master.
  • If preempt is enabled: whichever host starts advertising last (i.e., is configured last) will become the master.

The Demotion Counter

Another metric used in determining which host becomes master is the demotion counter. The demotion counter is a value advertised by CARP hosts that announce how “ready” a host is to take on the role of master.

The values used to calculate the demotion counter are stored in dynamic interface groups. By default, each CARP interface is a member of the carp interface group.

carp100: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
 carp: MASTER carpdev em0 vhid 100 advbase 1 advskew 0
 groups: carp
 inet netmask 0xffffff00 broadcast

The demotion value is viewed and set using ifconfig(8).

# ifconfig -g carp
carp: carp demote count 0
# ifconfig -g carp carpdemote 100
# ifconfig -g carp
carp: carp demote count 100

Here the demotion value of the carp interface group is set to 100.

When CARP advertises on the network, it takes the sum of the demotion values of all interface groups that the CARP interface is a member of and advertises that as the demotion counter. Hosts with higher values are less preferred to become master for that particular CARP group.

The demotion counter makes it possible to failover selected CARP groups rather than the “all or nothing” approach used with preemption.

Interface States

All CARP interfaces start in this state. Also, when a CARP interface is admin down it is put into this state. When a CARP interface is admin up, it immediately transitions to BACKUP. Note that in OpenBSD 3.8 and earlier, a bug exists which will cause the host to transition to MASTER right away if preempt is enabled.
The host is listening for advertisements from the master. If no advertisements are seen after 3 advertisement windows then assume the master is down, transition to MASTER state and start sending advertisements. If an advertisement is seen with a worse (higher) advertisement window than ours, and if preempt is enabled, transition to MASTER and start sending advertisements.
The host is forwarding traffic directed to the virtual/group IP address. The host is also sending advertisements once per advertisement window that announce its presence to other CARP hosts within the CARP group. The host still listens for advertisements from other CARP hosts. If an advertisement is seen with a better (lower or equal to ours) advertisement window, transition to BACKUP and allow the other host to become MASTER.

Note that changing any values associated with a CARP interface (timers, password, etc) will automatically result in the interface being put into the INIT state.

Under normal circumstances, there can be multiple hosts within a CARP group in the BACKUP state, but only one host will ever be in MASTER state.


Preempt Failover Race
In the following scenario, a race occurs: Two firewalls each connected to switches using two separate physical interfaces on the firewall. The firewalls have preempt enabled; fw01 is master, fw02 is backup on all CARP groups. One of the switches goes away causing the interfaces on the firewalls to go down. Since both firewalls have preempt enabled, they up their advskew to 240 on all remaining CARP groups. Here’s the race: Who becomes master now? There was an interesting discussion about this on the OpenBSD pf mailing list here: http://marc.theaimsgroup.com/?l=openbsd-pf&m=113881646826219&w=2. Steven S commented that his workaround is to only set preempt on the master firewall and to disable it on the other firewall(s). There is no permanent solution to this issue currently.