I like MPLS. And I don't necessarily mean as a solution to solve a problem, but as something to configure in the lab. It's fun to build things that do something when you're done. Setting up OSPF or EIGRP and being able to traceroute across routers is meh. But configuring MPLS with all the associated technologies — an IGP, LDP, MP-BGP, — and then getting all of them working in unison... when you get the traceroute working, it's rewarding.
Here's something to keep an eye out for when you're troubleshooting MPLS: An LFIB entry (that is, the Label Forwarding Information Base) that states "No Label" versus one that states "Pop Label". These mean very different things and can be the difference between a working Label Switched Path (LSP) and a non-working LSP.
Here's the topology I'm working with:
R21 and R8 are Customer Edge (CE) routers and they communicate through the MPLS network in the "BRANCHES" VRF. R4 and R7 are Provider Edge (PE) routers and R1, R5, and R6 are Provider (P) routers.
I'm going to examine traffic going from R8's 10.1.8.8 address and destined to R21's 10.1.21.21 address (in other words, traffic flowing counter-clockwise on the diagram). When R8 does a traceroute to R21, those packets have two MPLS labels applied to them:
- The inner label which is generated and advertised by R4 to the other PEs (in this case just R7) and is used by R4 to lookup which VRF the incoming packet should be routed in.
- The outer label which is used by every router in the MPLS network to get the packet hop-by-hop from R7 to R4.
The outer label gets the packet 90% of the way to the destination. It's typically swapped at every MPLS hop along the path. For instance, in my lab R6 is using label 18 to get packets to R4. When those packets reach R1, R1 swaps 18 with 16. This continues on down the line until the packet reaches the "penultimate router". Penultimate means "next to the last". Since R4 is the last router on the LSP between R7 and R4 that makes R5 the penultimate router.
In a working state, the penultimate router performs a pop of the outer label. It peels the outer label off, discards it, and forwards the packet onwards with just the inner — now the only — label. The ultimate router — in this case, R4 — receives this MPLS packet with the single label, does a lookup for the label in the LFIB and finds the VRF and adjacency information that it uses to forward the packet on to R21.
R4#show mpls forwarding-table labels 19 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 19 No Label 10.1.21.0/24[V] 8740 Tu0 10.4.4.21
The key to this being successful is that R5's LFIB entry for label 16 tells it to perform a Pop Label operation.
R5#show mpls forwarding-table labels 16 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 Pop Label 10.1.4.0/24 21716760 Et0/0 10.2.45.4
The non-working state is when R5's LFIB entry for label 16 instructs it to perform a No Label operation.
R5#show mpls forwarding-table labels 16 Local Outgoing Prefix Bytes Label Outgoing Next Hop Label Label or Tunnel Id Switched interface 16 No Label 10.1.4.0/24 21721500 Et0/0 10.2.45.4
No Label literally means to strip all of the MPLS labels off the packet and forward the raw IP packet (in this case it's IP).
What's wrong with this picture now?
R4 receives a regular IP packet and does a lookup in the FIB for 10.1.21.21.
R4#show ip cef 10.1.21.21 0.0.0.0/0 no route
The problem is that the ingress interface on R4 — eth0/1 — is in the global VRF but 10.1.21.21 exists in the "BRANCHES" VRF so the route lookup fails and end-to-end connectivity is broken.
R8#ping 10.1.21.21 source 10.1.8.8 timeout 1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.1.21.21, timeout is 1 seconds: Packet sent with a source address of 10.1.8.8 ..... Success rate is 0 percent (0/5)
Why is R5 choosing No Label instead of Pop Label?
The short answer: because R4 stopped advertising a label to R5.
The Pop Label action is actually signalled from the ultimate router (R4) to the penultimate router (R5). Put another way: R4 tells R5 to perform a pop. It does this by advertising a special label value called the "implicit-null" label. When R5 receives this, it knows it needs to perform a pop operation. If R5 doesn't receive this label advertisement, it has no choice but to pull all labels off and send the raw IP packet.
If you sniff the wire, you'll see that the implicit-null label actually has a value of "3". This is also set forth in RFC 3032 - MPLS Label Stack Encoding, section 2.1.
Possible reasons that R5 doesn't receive the implicit-null label from R4:
- LDP not running between R4/R5
- LDP neighborship not established between R4/R5
- R4 filtering outgoing label advertisements ("mpls ldp advertise-labels")
- R5 filtering incoming label advertisements ("mpls ldp neighbor label accept")
- CEF not enabled (Thanks @Rati)
When troubleshooting a broken LSP, remember to check the LFIB entries very carefully and pay attention to any entries with "No Label" as the outgoing label.
Disclaimer: The opinions and information expressed in this blog article are my own and not necessarily those of Cisco Systems.