Continuing from our previous post, we will now see how BGP PIC EDGE works. We will use the same topology. The only difference is that I have removed R4 as route-reflector. All the PEs have full mesh IBGP neighbourship.
On R2
R2 now knows that R6 is unavailable so it will pick 7.7.7.7 as the next-hop.
We have a similar problem which we face for PIC Core. However in this case the BGP neighbourship goes down so the issue is more significant. As the IBGP next-hop IP changes, we cannot resolve this problem with only PIC Core.
In normal condition if the BGP table has 500K routes, it would take few minutes for BGP to reconverge completely.
In this scenario, we can speed up the convergence if we can install two paths in BGP table and FIB.
PIC EDGE can help in this situation. There are various ways to enable it.
1. One of the ways is by using the command "maximum-paths".Let's configure it on R2.
If we enable R6 and check R2's BGP table, we can see that now have multipath entry.
Even in the FIB, we can see two entries.
Now if R2 loses the path through R6, it will pick up the path through R7. As the entry is already written in FIB, there will be no need to rewrite all the BGP routes one by one.
2. The second way of enabling PIC EDGE is by using "advertise-best-external & Add-Path". Let's see how it works.
I have changed the configuration and applied local preference of 200 on the path via R6.
We can now see that even though we have applied the "maximum-path" command on R2, it selects the best path through R6 only.
What is the problem here? Looking at R7, we can see it prefers 8.8.8.8/32 via R6 due to local preference hence it stops advertising the route via it's EBGP neighbour (10.1.78.8).
It would help if R7 keeps advertising the backup route it learned via EBGP to R2. The way we can make it happen is by configuring "advertise-best-external" in R7.
We can now see that R7 treats the backup path as "best-external".
If we look at R2, we can see that it has started receiving the backup path from R7 now.
However the FIB still shows one possible path via R6.
Here we will have to enable the feature called "Add-Path".
"bgp additional-paths select backup" calculates a second bestpath as the backup path.
"bgp additional-paths install" will install the type of path that is specified in the "bgp additional-paths select" command.
If we check the FIB on R2,
We still don't see the second path!!!
Be Careful!! This is where Cisco IOS can trick you!
We need to use the "detail" keyword to see the backup path in FIB.
R2 has installed a backup "repair" path in FIB via R7.
So far we have seen how PIC EDGE would work in full mesh IBGP environment but what if we have route-reflectors in the network.
I have changed the configuration of the routers and removed the full mesh IBGP between PEs. The router R4 is now configured as VPNv4 route-reflector. I have also removed the local preference on R6 and "Add-Path" commands on relevant PEs.
Ok, so we can see that R4 receives two paths to reach 8.8.8.8/32. It selects the path via R6 as the best path.
R2 receives the best path information from R4.
The problem here is that route reflector will always chose the best path and only advertise that to it's clients which will make multipathing impossible.
What if we add another route-reflector in the network? Normally in real world most of the networks do have more than one route-reflectors.
Let's configure R5 as route-reflector.
Configure all the PEs with required config. I have just shown the config of R2 for brevity.
If we check R2, We can see that it has started receiving 2 paths from R4 and R5.
Can we use Add-path to do multipathing?
Remember, Add-path will not work if the next-hop of backup path is the same as primary path. The next hop must be different. In our case, the next-hop advertised by both the RRs is 6.6.6.6!
So, the problem can be resolved if R5 can advertise the path via R7 (7.7.7.7) instead of R6.
3. The solution is to use "Diverse-path" along with "advertise-best-external" and "Add-Path".
On R5, we have configured ""Add-path" and "diverse-path". This will make R5 to install the backup path in the BGP table and advertise it to the specific neighbour R2.
R2 has now started receiving a path with the next-hop 7.7.7.7 from R5.
We can now configure "Add-Path" on R2.
The backup path is installed in the FIB.
At the moment, the route advertised by R6 and R7 has the same local preference of 100. What if I change the config and prefer the route via R6 by increasing the local preference.
Suddenly, R7 has started advertising the path via R6 (6.6.6.6) to R2 which will break the "Add-Path" functionality.
What happened here?? Let's see step by step
1. R6 received the route 8.8.8.8/32 and advertised to RRs(R4 and R5).
2. R7 received the route 8.8.8.8/32 and advertised to RR2(R4 and R5).
3. R5 started calculating it's second best path.
4. Meanwhile R7 started learning about the path with higher local preference via R6 from R4.
5. R7 withdraws it's advertisement of 8.8.8.8 from R4 and R5 as it has a better path via R6 through higher local preference.
6. Now R5 calculates the only path it receives from R6 and pass it on to R2.
As per above output, we can see that R7 prefer the path through R6.
How can we force R7 to advertise it's backup EBGP path to RRs?
As previously seen in our post, we can use "advertise-best-external".
On R2, the backup path is now installed in the FIB.
More information can be found on http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-3s/irg-xe-3s-book/irg-bgp-mp-pic.html
At the moment R2 learns 8.8.8.8/32 from R6 and R7. It prefers the path with the next-hop of 6.6.6.6 over 7.7.7.7.
I have removed the OSPF cost 1000 on R2's interface towards R4 so at the moment R2 has two equal cost path to reach 6.6.6.6.
Now, What will happen if R6 goes down?
R2 now knows that R6 is unavailable so it will pick 7.7.7.7 as the next-hop.
We have a similar problem which we face for PIC Core. However in this case the BGP neighbourship goes down so the issue is more significant. As the IBGP next-hop IP changes, we cannot resolve this problem with only PIC Core.
In normal condition if the BGP table has 500K routes, it would take few minutes for BGP to reconverge completely.
In this scenario, we can speed up the convergence if we can install two paths in BGP table and FIB.
PIC EDGE can help in this situation. There are various ways to enable it.
1. One of the ways is by using the command "maximum-paths".Let's configure it on R2.
If we enable R6 and check R2's BGP table, we can see that now have multipath entry.
Even in the FIB, we can see two entries.
Now if R2 loses the path through R6, it will pick up the path through R7. As the entry is already written in FIB, there will be no need to rewrite all the BGP routes one by one.
2. The second way of enabling PIC EDGE is by using "advertise-best-external & Add-Path". Let's see how it works.
I have changed the configuration and applied local preference of 200 on the path via R6.
We can now see that even though we have applied the "maximum-path" command on R2, it selects the best path through R6 only.
What is the problem here? Looking at R7, we can see it prefers 8.8.8.8/32 via R6 due to local preference hence it stops advertising the route via it's EBGP neighbour (10.1.78.8).
It would help if R7 keeps advertising the backup route it learned via EBGP to R2. The way we can make it happen is by configuring "advertise-best-external" in R7.
We can now see that R7 treats the backup path as "best-external".
If we look at R2, we can see that it has started receiving the backup path from R7 now.
However the FIB still shows one possible path via R6.
Here we will have to enable the feature called "Add-Path".
"bgp additional-paths select backup" calculates a second bestpath as the backup path.
"bgp additional-paths install" will install the type of path that is specified in the "bgp additional-paths select" command.
If we check the FIB on R2,
We still don't see the second path!!!
Be Careful!! This is where Cisco IOS can trick you!
We need to use the "detail" keyword to see the backup path in FIB.
R2 has installed a backup "repair" path in FIB via R7.
So far we have seen how PIC EDGE would work in full mesh IBGP environment but what if we have route-reflectors in the network.
I have changed the configuration of the routers and removed the full mesh IBGP between PEs. The router R4 is now configured as VPNv4 route-reflector. I have also removed the local preference on R6 and "Add-Path" commands on relevant PEs.
Ok, so we can see that R4 receives two paths to reach 8.8.8.8/32. It selects the path via R6 as the best path.
R2 receives the best path information from R4.
The problem here is that route reflector will always chose the best path and only advertise that to it's clients which will make multipathing impossible.
What if we add another route-reflector in the network? Normally in real world most of the networks do have more than one route-reflectors.
Let's configure R5 as route-reflector.
Configure all the PEs with required config. I have just shown the config of R2 for brevity.
If we check R2, We can see that it has started receiving 2 paths from R4 and R5.
Can we use Add-path to do multipathing?
Remember, Add-path will not work if the next-hop of backup path is the same as primary path. The next hop must be different. In our case, the next-hop advertised by both the RRs is 6.6.6.6!
So, the problem can be resolved if R5 can advertise the path via R7 (7.7.7.7) instead of R6.
3. The solution is to use "Diverse-path" along with "advertise-best-external" and "Add-Path".
On R5, we have configured ""Add-path" and "diverse-path". This will make R5 to install the backup path in the BGP table and advertise it to the specific neighbour R2.
R2 has now started receiving a path with the next-hop 7.7.7.7 from R5.
We can now configure "Add-Path" on R2.
The backup path is installed in the FIB.
At the moment, the route advertised by R6 and R7 has the same local preference of 100. What if I change the config and prefer the route via R6 by increasing the local preference.
Suddenly, R7 has started advertising the path via R6 (6.6.6.6) to R2 which will break the "Add-Path" functionality.
What happened here?? Let's see step by step
1. R6 received the route 8.8.8.8/32 and advertised to RRs(R4 and R5).
2. R7 received the route 8.8.8.8/32 and advertised to RR2(R4 and R5).
3. R5 started calculating it's second best path.
4. Meanwhile R7 started learning about the path with higher local preference via R6 from R4.
5. R7 withdraws it's advertisement of 8.8.8.8 from R4 and R5 as it has a better path via R6 through higher local preference.
6. Now R5 calculates the only path it receives from R6 and pass it on to R2.
As per above output, we can see that R7 prefer the path through R6.
How can we force R7 to advertise it's backup EBGP path to RRs?
As previously seen in our post, we can use "advertise-best-external".
On R2, the backup path is now installed in the FIB.
More information can be found on http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-3s/irg-xe-3s-book/irg-bgp-mp-pic.html
Excellent. gone through again with PIC core and edge. Make more sense when IGP fails with more BGP routes and when BGP fails with lot of BGP routes. Thanks for nice and simple post to clear couple of ways to implement PIC edge. Thank you
ReplyDeleteThe article is concise and excellent. well explained by unknown??
ReplyDelete