Troubleshooting Cisco Network Elements with the USE Method

I want to draw some attention to a new document I’ve written titled “Troubleshooting Cisco Network Elements with the USE Method“. In it, I explain how I’ve taken a model for troubleshooting a complex system–the USE Method, by Brendan Gregg–and applied it to Cisco network devices. By applying the USE Method, a network engineer can perform methodical troubleshooting of a network element in order to determine why the NE is not performing/acting/functioning as it should.

I ask that if you’re familiar with a given Cisco network platform (or platforms), that you please contribute commands that would also fit into the USE Method! My list is just a start and I welcome contributions from others in order to make it a stronger, more valuable reference.

Please check out the guide: Troubleshooting Cisco Network Elements with the USE Method

Tools for TE with EIGRP

In response to my article about what would cause a directly connected route to be overridden, Matt Love (@showflogi) made a good observation:

What Matt is saying is that longest prefix match (LPM) is a mechanism that can be used to steer traffic around the network in order to meet a technical or business need. This type of traffic steering is called traffic engineering (TE). Continue reading Tools for TE with EIGRP

When is a Connected Route Not Used?

I ran into this situation on a recent project and thought it would make an excellent question on an exam. It could be worded something like this:

What is the behavior of a router or Layer 3 switch when a dynamic route is learned that partially overlaps with a directly connected network?

a. The router reboots
b. The network reboots
c. That’s um-possible
d. None of the above

Continue reading When is a Connected Route Not Used?

Reflecting On My First Cisco Live! Presentation

Well, I got to tick a big item off my list of goals last week. I successfully delivered a presentation at Cisco Live! in front of a large group of people. It didn’t kill me and I didn’t trip over anything and embarrass myself so no matter what, I have those two points to feel good about :-)

Me starting my presentation
Me starting my presentation

All joking aside, it actually went a whole lot better than that. Continue reading Reflecting On My First Cisco Live! Presentation

OpenVPN 2.3.17 on OpenBSD 6.0

On Jun 21, the OpenVPN team released an update for the 2.3.x and 2.4.x branches that resolved some newly discovered security vulnerabilities. The OpenVPN team recommends that users “upgrade to OpenVPN 2.4.3 or 2.3.17 as soon as possible“.

OpenBSD 6.0–which was released Sep 1 2016 and is still receiving security updates to the base system as per OpenBSD’s policy–shipped with a package for OpenVPN 2.3.11. Below you will find a patch and instructions for using the ports system to upgrade to version 2.3.11. Note that if you’re running OpenBSD 6.1, the ports tree has been updated to 2.4.3 so all you need to do is “cvs up” and “make install”.

Instructions:

  1. Follow the OpenBSD FAQ for instructions on how to download, verify, and extract the ports tree on your machine.
  2. Then:
% cd ports/net/openvpn
% patch < ~/openvpn-2.3.17p0.diff
% make install

I Will Be Presenting For the First Time at CLUS 2017!

Well, it looks like another major item will get struck from my bucket list this year. I’ve been accepted to present at Cisco Live in Las Vegas this summer! 👊

This session is designed to walk through an enterprise network and look at how EIGRP can be engineered with purpose to best suit the needs of the different areas of the network. I will focus a lot on stability and scaling EIGRP and will show the audience how, where, and when to leverage common EIGRP features such as summarization, fast timers, BFD, and wide metrics. Before getting into the nuts and bolts, I will be doing a bit of a level-set on certain EIGRP features such as queries, going active, summarization, and support for flexible network hierarchies. I will round out the session by talking about how EIGRP has been optimized for use in Cisco’s Intelligent WAN (IWAN) solution and even touch on a not-so-commonly seen application of EIGRP: EIGRP Over-The-Top. The full session agenda looks like this:

I’m actually inheriting this session from a fellow CPOC engineer, Steve Moore who, un-coincidentally, is the same S. Moore whose name is on the EIGRP RFC. Steve will be presenting a sister session the following day: BRKRST-2331: Troubleshooting EIGRP Networks.

If you’re reading this and you’re coming to CLUS this year, be sure to register and come listen to one or both of these sessions and pick up some EIGRP knowledge!

  • BRKRST-2336 — EIGRP Deployment in Modern Networks: Tue June 27 1:30pm – 3:30pm
  • BRKRST-2331 — Troubleshooting EIGRP Networks: Wed June 28, 1:30pm – 3:30pm

Five Functional Facts About OSPF

It’s funny, in my exerperience, OSPF is the most widely used interior gateway protocol because it “just works” and it’s an IETF standard which means it interops between different vendors and platforms. However, if you really start to look at how OSPF works, you realize it’s actually a highly complex protocol. So on the one hand you get a protocol that likely works across your whole environment, regardless of vendor/platform, but on the other you’re implementing a lot of complexity in your control plane which may not be intuitive to troubleshoot.

This post isn’t a judgement about OSPF or link-state protocols in general. Instead it will detail five functional aspects of OSPF in order to reveal–at least in part–how this protocol works, and indirectly, some of the complexity lying under the hood.

Continue reading Five Functional Facts About OSPF

Why I Enthusiastically Switched from Cacti to Zabbix for System Monitoring

Cacti is a “complete network graphing solution” according to their website. It has also been a thorn in my side for a long time.

See what I did there? Thorn… because it’s a cactus… never mind.

When Cacti is in a steady state–when I could get it to a steady state–it was good. Not great, because there was a lot of effort to get it into what I consider “steady state”, but good. The rest of the time… thorny.

There are five major things that have driven me up the wall. In no particular order: Continue reading Why I Enthusiastically Switched from Cacti to Zabbix for System Monitoring

Big Changes in 2017

This past June when I was in North Carolina at Cisco’s CPOC lab, I learned that there was a chance–albeit a slim one, but a chance nonetheless–that a position would be opening up on the CPOC team in the fall. By that point I had been to CPOC three times and knew many of the engineers who worked there. I spoke to them to get their feedback, met with the newly-hired manager of the team, and just generally did all the things I thought I should be doing to take advantage of my time being face to face with these folks.

Then I flew home, subscribed to the “new jobs at Cisco mailing list” and waited.

And then, one day, it was posted: CPOC Technical Projects Systems Engineer. I immediately sent a message to my wife who responded as only she knows how:

Val_CPOC_job_reaction.png
Excitement :-)

Five short interviews later I was offered the job!

This brings me to change #1: As of this month (January), I am no longer a Systems Engineer with Cisco Systems Canada. I am now a Systems Engineer on the CPOC team reporting to a manager in the US.

Beyond the basic level of excitement I have about joining this team, I’m even more excited because I’m being hired for a role that isn’t quite the typical CPOC engineer role. My role is part of an initiative to help field sales teams (which up until today, would’ve included myself) sell various Cisco enterprise networking solutions, the first of which is Cisco Intelligent WAN. My role in this initiative is to provide technical expertise in helping field SEs conduct proof of value (POV) exercises with their customers through a combination of remote support and direct hands-on engagement at customer locations (as the situation warrants).

This is a new initiative at Cisco and involves a lot of people at different levels and in different parts of the company. I’m really excited to get back to a more technical, hands-on role, to feed my field experience back into this initiative to make it successful and valuable to the field teams, and to be involved in an initiative this big from the ground floor.

Now, you may have noticed that I said above that I am reporting to a manager in the US. Well, that’s because the CPOC lab is in RTP, North Carolina and that’s where the team is based.

That brings me to change #2: My wife and I (and the cats, can’t forget the cats 😼) will be moving from Calgary to the Raleigh/Durham area in the coming weeks.

The Raleigh area is green. Like, REALLY green.
The Raleigh area is green. Like, REALLY green. [Pic after takeoff from RDU]
Something else we did when I was at CPOC in June was to make sure my wife came down over the weekend so she could (finally!) see the area and understand why I loved going down there so much. It took her no time at all to get it. Now that we’re looking at moving, she understands the area a bit and has a feeling for what we’ll be stepping into.

She grew up and has lived her whole life in Calgary and I grew up not far from Calgary and have been here for about 15 years. We’re very excited for this change! We’re excited for a different climate, we’re excited to explore, and we’re excited to be close(r) to the ocean.

So there we go! Not a long list, but a very impactful one, for sure. Thank you to everyone for your support, small and large, over the past few and upcoming weeks! Bring on the BBQ!

Update Feb 22 2017

It’s been a few weeks since I posted that I had taken a new job and that we were moving to North Carolina. A lot has happened since then, enough that I thought it warranted this unplanned update.

First, I’ve been doing the new job since early January and it’s been a lot of fun. Normally a Cisco CPOC engagement means the customer comes to Cisco and does their testing in our lab, but I was hired to take the show on the road and conduct demonstrations on-site with customers. It’s been a lot of fun playing a part in getting this new type of CPOC delivery up and going.

Second, there has been a rather major, unexpected wrinkle in the relocation. While reviewing my tax situation with a professional tax advisor, I came to learn about a Canadian tax known as the departure tax. This tax kicks in once you become a non-resident in the eyes of the Canada Revenue Agency (CRA). CRA has a set of criteria for determining who is a resident, none of which we would meet. As a non-resident, we would be hit with the departure tax.

The departure tax is basically CRA’s way of ensuring they get what they feel they’re entitled to in terms of taxes on capital gains and other investments. The reasoning seems to be that if you leave Canada, you’re untouchable from CRA’s perspective and they won’t be able to collect tax from you. The departure tax is very odd though because you don’t actually have to sell anything or realize any actual capital gains for the tax to kick in. What happens is something called a “deemed sale” where–for all intents and purposes–everyone pretends that you executed a sale and you are summarily taxed on the imaginary capital gains. Beyond just capital gains, there are other investment assets that are also taxed.

What hurts about this is that it all comes in one shot. And in our situation, having never heard about this before now, we had not done any financial planning in order to mitigate the tax hit. Our exposure to this is rather high and being that it comes all in one shot, would be very difficult to absorb.

Because of this tax situation, the move is now on hold until further notice. Thankfully, my role on the CPOC team allows me to work from anywhere and my manager has been absolutely amazing in her understanding and patience.

So for now, my wife and I are digging our toques and mittens out of storage, mentally preparing to have to continue dealing with winter, and making plans for the future to cut our exposure to this little known, high impact part of the Canadian tax code.

 

Networking. Unix. Cyber Security. Code. Protocols. System Design.