VRFs and Shared Services Cheating with Junos

The shared services area of the network is meant to provide common services — such as DNS, DHCP, and Internet access — to multiple logical networks/VRFs/customers. Cisco publishes a validated design for shared services that describes the use of multiple virtual firewalls and routers to provide connectivity between the shared services module and the VRFs in the network. I’m going to describe a method of collapsing the shared services firewalls and virtual routers into a single instance running on a single box using some of the features found in Juniper’s Junos platform.

This is the third post in my series on virtual routing. Other articles in the series are

Shared Services Objectives

The shared services module has 3 main objectives:

  1. Provide shared network services to multiple VRFs by way of common infrastructure
  2. Permit select traffic exchange between VRFs and shared services resources
  3. Prevent traffic exchange between VRFs

The last one isn’t a reason for setting up a shared services module (ie, you don’t setup shared services in order to try and keep traffic from passing between VRFs) but instead is a very important design objective. Since the shared services module has connections into multiple VRFs and is exchanging routes with them, due care is needed to prevent the shared services area from becoming a transit path between VRFs.

Now, you might think that since you’re connecting your VRFs together via a common area that you’re increasing your risk of traffic crossing between VRFs (ie, guests being able to access your corporate network resources). The natural reaction is to stuff some firewalls in the mix. Filter everything in and out of the VRFs! Firewalls will keep us safe! Keep in mind though that each VRF only forwards traffic toward a destination if it has a route for that destination. As long as routes are being filtered properly in the shared services area, VRF A will not contain routes for networks in VRF B (and vice-versa). If a user sends a packet into VRF A with a destination address belonging to a network in VRF B, the worst case is that the packet will follow the default route in VRF A (which probably leads towards the Internet exit point in that VRF) and best case it will just be dropped because there is no matching route. Either way, even without a firewall, traffic does not cross from A to B.

Remember: Traffic will not be directed towards (or through) the shared services area unless there is a matching route present in a VRF’s routing table that points towards the shared services area.

Having said that, we can’t throw the firewall out the window. Objective #2 is to provide selective traffic exchange between VRFs and the shared services and that’s where the firewalls come in. They will be used to permit only the traffic that corresponds to the specific services that are being provided. For example, if one of the shared services is DNS, only allowing port 53 traffic into the shared services area.

Cheating

The basic topology for this shared services module is shown here:

The consolidated router in this topology is actually a Juniper SRX firewall running Junos and will be performing route exchange, route filtering, and firewall duties all in the same physical box and in a single firewall instance.

The consolidated router is configured with one routing instance for each VRF in the network. Each instance is of type “virtual-router” and is used to exchange routes with the VRFs in the network as well as containerize those routes.

By importing explicit prefixes between each routing instance and the global, aka “master” routing instance, route filtering is achieved. The OSPF process running within each routing instance is then directed to pick up these imported prefixes and advertise them to its neighbors (which are OSPF processes running in the VRFs on the edge routers).

OSPF has no ability to filter outgoing route updates. The edge routers could be configured with distribute lists to prevent incoming route updates from entering their routing tables however this would not prevent the prefixes from being present in the OSPF link state database. More importantly, this method creates two points of management for route filtering vs. a single point on the consolidated router.

BGP may be a better protocol to use between the consolidated router and the edge routers as it would more easily allow filtering of sent and received prefixes. It would probably remove the need for routing instances at all (thus making the design more in line with Cisco’s validated design of having a single routing table in the shared services area). However I’m basing this post on a real life deployment which actually used OSPF.

 By employing routing instances on the consolidated router it’s possible to:

  • Accurately filter routes in and out of the shared services area
  • Extend and reuse the concepts of virtualized routing tables which are already being used in the core of the network (and therefore something we’re already familiar with)
  • Avoid the need to manage route filtering in more than one location; everything is done on the consolidated router

On top of this, a firewall policy will be deployed that permits only specific traffic based on typical 5-tuple information. So even though a VRF has full routes for the shared services area, the firewall policy will control exactly what traffic is allowed into (and out of) the area.

The Configuration

For brevity, I’m only going to show the configuration related to the blue VRF/routing instance. I’m also not going to show the firewall security policy as this post is meant to focus on the routing configuration.

First, configure the instance.

routing-instances {
    blue {
        instance-type virtual-router;
        interface ge-2/0/1.0;
        interface ge-3/0/1.0;
        protocols {
            ospf {
                area 0.0.0.0 {
                    interface ge-2/0/1.0;
                    interface ge-3/0/1.0;
                }
            }
        }
    }
}

This gives us a basic virtual-router instance with two interfaces bound to it and OSPF running within it. The edge routers should be configured similarly with with their respective interfaces bound to the blue VRF and OSPF enabled.

The two interfaces ge-2/0/1.0 and ge-3/0/1.0 face the edge routers and are configured as regular point-to-point /30 interfaces.

interfaces {
    ge-2/0/1 {
        description "Connects to Edge Router #1";
        unit 0 {
            family inet {
                address 10.2.0.1/30;
            }
        }
    }
}

At this point the blue routing instance should contain all of the routes from the blue VRF.

The interface facing the shared services area is a regular interface with no associated routing instance; it’s part of the default instance (named “master” on the SRX platform).

interfaces {
    ge-2/0/0 {
        description "Connects to shared services switch";
        unit 0 {
            family inet {
                address 10.10.0.1/24;
            }
        }
    }
}

At this point the “master” routing instance only contains the “direct” route for the /24 network configured on ge-2/0/0.0.

Now, to import routes into “master” from “blue”, configure an appropriate policy statement that selects routes in “blue” and apply that statement as an import policy on the “master” routing instance.

policy-options {
    policy-statement routes_from_vrfs_to_master {
        term from_blue {
            from {
                instance blue;
            }
            then accept;
        }
        term default_deny {
            then reject;
        }
    }
}
routing-options {
     instance-import routes_from_vrfs_to_master;
}

This config will suck all the routes from the “blue” routing instance into the “master” instance. To be a little more granular, a prefix-list could be used in the “from” stanza of the policy-statement to only permit specific prefixes but that would limit reachability to only a portion of the blue network. The default_deny term is important; you’ll want to make sure that’s the last term in the policy-statement.

Now to do the reverse and import routes into blue from master. This is where the filtering becomes relevant.

policy-options {
    prefix-list ss_prefixes {
        10.10.0.1/24;
    }
}

This prefix-list specifies the network(s) that belong to the shared services area and is used to filter the list of routes imported into blue.

policy-options {
    policy-statement routes_from_ss_to_vrfs {
        term 1 {
            from {
                instance master;
                prefix-list ss_prefixes;
            }
            then accept;
        }
        term default_deny {
            then reject;
        }
    }
}

The above chunk selects prefixes in the master table that match the prefix-list.

routing-instances {
    blue {
        routing-options {
            instance-import routes_from_ss_to_vrfs;
         }
        protocols {
            ospf {
                export routes_from_ss_to_vrfs;
            }
        }
    }
}

And this chunk applies the policy in order to:

  • Import the routes into blue from master
  • Export the routes out of blue to OSPF so that the prefixes are advertised to the edge routers

At this point traffic should flow bidirectionally between shared services and the “blue” VRF while at the same time preventing traffic flow between “blue” and any other VRF.

15 thoughts on “VRFs and Shared Services Cheating with Junos”

  1. Very nice article!!

    But there is a question that I am thinking about:
    Let say 10.10.10.0/24 is subnet present in all three VRFS, machines on these subnet in all three VRF have a need to communicate ” Shared Service” on 10.10.0.0/24

    The issue will be for the traffic flowing from ” Shared Service” to 10.10.10.0/24 subnet which exists in all three VRF. How does Consolidated router use main routing table to direct the traffic to appropriate VRF because 10.10.10.0/24 is ” sucked” into the main routing table from all three VRFS on Consolidated router because of IMPORT POLICY?

    Much appreciated!!

    1. Thanks for the feedback!

      In the scenario you describe, you’d have to use NAT to translate that subnet into address space that is unique inside the master/global table.

      1. Can you please provide some explanation with blog so that I will understand it better. I tried with destination NAT and failed.

          1. Hi Joel,

            Not like that. Don’t mistake me. As Zeeshan said, we have two customers having the same IP subnets and are in different VRF’s. And there is one shared VRF which has some routes required for both the customers. But the shared VRF not able to route the return traffic to the appropriate customers.

            I am using instance-import configuration for the route leaking. But only one customer interface IP’s are importing into the shared vrf(as both cust are having same IP). I tried using destination NAT with from zone as shared. but no success :( .

            I think am missing something in my config.

            Cust 1 : 10.10.10.1/29 (VRF : CUST1) .2 and .3 are usable
            Cust 2 : 10.10.10.1/29 (VRF : CUST 2) .2 and .3 are usable
            NAT for cust 1 10.10.10.2 to 1.1.1.1 and 10.10.10.3 to 1.1.1.2
            Nat for Cust 2 10.10.10.2 to 2.2.2.1 and 10.10.10.3 to 2.2.2.2

            1. Hi Anand,

              Ok, I understand now that you have overlapping address space. Like I said to Zeeshan, you would have to use NAT in a situation like this to hide the 10.10.10.0/29 address space from the shared services VRF.

              You said you tried “destination NAT”. Are you trying to setup NAT on the consolidation router for traffic initiated from shared services and destined to the respective VRFs? Does the shared services VRF initiate traffic towards a customer? Would it make more sense to have a source NAT to NAT the source address when traffic moves from customer->shared services? Or is dest NAT really what you need?

              You also mentioned you tried “from zone shared” which hints to me that you’re doing NAT on the consolidation device. That’s a lot of complexity being put on that device (inter-VRF routing, route leaking, access control, and NAT). Have you considered moving NAT back to the edge router(s)? I’m not suggesting this would fix the problem you’re seeing, but just a general comment.

              What troubleshooting have you done with the NAT? Have you taken any captures to see what the src/dest addresses are on packets on both sides of the consolidation device? Is the consolidation device a firewall? Do you have an order of operations issue between NAT and the firewall policy?

              1. I try to brief you a bit..

                Cust A – Using the IP range 10.10.10.0/29. In that range only two servers are available (for example win and linux). And those IP’s are win-10.10.10.2 and linux-10.10.10.3. And 10.10.10.1 is a gateway(interface IP in srx)

                Similarly for Cust B. But the difference is NAT IP. Cust A NAT IP’s (win-1.1.1.1 and Lin-1.1.1.2) and Cust B Nat(Win-2.2.2.1 and Linux- 2.2.2.2).

                Both customers acessing some application which are outside our datacenter via shared VRF. We have a static route for the applications and those static routes are imported into respective customer VRF’s using import-instance route-leaking concept.

                So when the traffic is initiated from Cust A or Cust B towards the application, source nat is applied and the connectivity is fine. No issues am facing.

                The zones for these are :
                Cust A
                Cust B
                Shared

                So, the source NAT has been configured as from zone Cust A to zone Shared. Similarly for Cust B. Everthing works fine.

                But the problem is when the traffic is initiated from outside. (For ex: VPN customer access the Cust A or Cust B from outside via Shared VRF with the NAT IP). So I need to configure the destination NAT .
                Traffic reaching the shared VRF with destination IP(Cust A or Cust B NAT IP) Here only I am get stucked. If I do a destination NAT in shared zone, the NAT IP has been changed to cust server IP(which is common for both the customers). So it doesn’t know where to route the traffic. I tried in configuring the dest NAT with from zone as Cust A or Cust B but we need a route for NAT IP in shared VRF. I don’t know how the NAT IP routes will be configured in shared VRF.
                As the route lookup happens after the destination NAT, i am little confused on how to configure here to sort it out. This problem happens only when I do route-leaking with the instance-import concept. If I do route leaking via lt concept. Everything works fine. I send you the logs once I again get a downtime to check the same.

                1. Hi Anand, that’s a very detailed explanation. Thank you.

                  I understand now that the route lookup happens after dest NAT and so you have an order of operations issue.

                  Can you explain the lt concept of routing leaking? What’s the reason you cannot use that as the solution?

                  Is it an option to move the NAT back further into the customer’s network? Do the customer networks have an L3 device that connects to the SRX? You’d want to assign unique /29s (or larger) to the interconnect and then have the customer device NAT 10.10.10.2 and .3 to specific IPs in the /29. This would solve the overlapping address space issue and the order of operations problem on the SRX.

                  If this was Cisco IOS, I might look at VRF-aware NAT to see if the NAT feature-set could push the incoming shared packets into the customer’s VRF at the same time it’s doing translation.

                  1. I am using lt here for the route leaking with ibgp protocol.

                    For ex: I created lt-0/0/0.1 with peer-unit as 2 and lt-0/0/0.2 with peer unit as 1. Then I bind lt-0/0/0.1 in cust A vrf and lt-0/0/0.2 in shared vrf routing instance. And I created three zones namely, CUST-Zone, Interconnect-Zone and Shared-Zone.

                    In shared-zone I include the interface lt-0/0/0.2 and in Interconnect-zone I included lt-0/0/0.1. Coming to destination NAT, I configured from zone as Interconnect-zone and the NAT is working perfectly without any issues. When establishing ibgp between cust and shared vrf, I exported the NAT IP’s in iBGP configuration from cust to shared, so that shared vrf is having the route for NAT IP and also I configured the discard static route in cust VRF for the NAT IP to avoid the route loop.

                    But I want this to be achieved without lt interface. But I am unable to do that :(

                    1. Interesting feature, that lt interface.

                      This problem is now way beyond my ability with JunOS, unfortunately. Maybe you want to try posting in the Juniper forums or talking to JTAC.

  2. Thanks Joel for your time and kind response. I will try to find out the answer. If you get anything, please comment me :)

Leave a Reply

Your email address will not be published. Required fields are marked *

Would you like to subscribe to email notification of new comments? You can also subscribe without commenting.