Ingress BGP Routing Using Communities & Local Preference

Influencing ingress BGP routing using communities and local preference

This blog explores a lesser-known but powerful traffic engineering technique using BGP communities and the local-preference value.

Author: Nicholas Russo


Border Gateway Protocol (BGP) is an enormous protocol with a nearly endless list of features, knobs and capabilities. BGP’s mechanism for choosing the best path is complex but also well known. You should brush up on that algorithm if you’re out of practice. This blog explores a lesser-known but powerful traffic engineering technique using BGP communities and the local-preference value. You should have basic familiarity with BGP and IP routing in general to realize the most value from this blog.

Consider the following topology. R1 is a small business with two Internet uplinks, one through R2 and one through R4. R2 provides transit service for R1 into AS 65345, a large regional carrier. R1’s link to R4 is much slower (or higher latency, higher monetary cost, lower reliability, etc.) and should only be used as a backup. This policy decision should be implemented bidirectionally, applying to traffic egressing from R1 and for traffic ingressing to R1. I’m a big IPv6 fan and this network is IPv6-only. All the same concepts apply to IPv4 and dual-stacked networks as well. The full configurations for each device are at the end of this blog for those interested. Last, I’m using the RFC 3849 IPv6 documentation prefix to avoid any accidental overlap with production networks.

Handling R1’s egress traffic is straightforward and is easily achieved using local-preference applied inbound. As an example, R1 can apply a value of 100 (the default on Cisco routers) from R4 and apply a value of 200 from R2. Since higher values are preferred, all routers in AS 65001 will prefer R2 for egressing traffic towards AS 65345 and beyond. We won’t explore this configuration in-depth today.

Handling R1’s ingress is more complicated. Traditionally, there are three common solutions.

Use the Multi Exit Discriminator (MED)

This represents a BGP “metric” and hints at the internal IGP topology to a peer AS of the advertiser’s AS. In reality, many carriers ignore the MED value, and even if they honor it, it’s evaluated quite late in the BGP best-path algorithm. Worse still, it typically isn’t compared between different AS numbers, so the AS 65345 routers would not compare MED values from R1 and R2 directly. MED is not the correct tool for R1 in this scenario, as illustrated by the diagram below.

Use AS-path prepending

Slightly more effective than MED, this allows an advertiser to artificially lengthen the AS-path, making a prefix less desirable across a given link. In a simple case, this could solve R1’s problem, as R1 could just prepend 65001 on its advertisements towards R4 so that AS 65345 prefers R3 as the egress point towards R1 with R2 as an intermediate hop. However, this is not guaranteed. Suppose R2 is prepending AS numbers towards R3 to make R4 the preferred egress point from AS 65345! That would foil R1’s plan quite nicely. Also, consider a case where instead of a single AS, such as AS 65002, there is a large number of AS hops between AS 65001 and AS 65345. This would have the same effect on the AS-path length and leave R1 guessing as to the correct number of prepend actions. The diagram below illustrates a failed attempt by R1 to use this technique.

Influencing ingress BGP routing using communities and local preference

Use longest-match routing

Advertising more specific prefixes, such as a single /127 via the backup link and two /128s via the primary link, is a reliable technique to guarantee ingress, as illustrated below. Upstream carriers seldom aggregate or filter customer prefixes (unless they are allocated from a provider aggregate range, of course), but there is a drawback. For IPv4, /24 is the shortest prefix allowed on the Internet, and for IPv6, /48 is the shortest. If all you have is an IPv6 /48, you can’t carve it into two /49s without a special arrangement with your upstream ISPs (good luck with that). Let’s assume that this technique is not currently available to Globomantics.

Influencing ingress BGP routing using communities and local preference

So, what is our customer to do? This is a common issue and has vexed many businesses. In response to this, service providers have developed an alternative solution that relies on BGP communities. Communities are like sticky notes; they can be attached to various routes to signal arbitrary attributes of a route. A service provider can match a given community for a received route, then set the local-preference to a corresponding value. In plain English, the customer says, “I want you to set local-preference X on this prefix, so I’ve attached community Y to signal that”. Implementations will inevitably vary between vendors, but broadly speaking, a giant if/then statement is evaluated to map community values to local-preference values.

Influencing ingress BGP routing using communities and local preference

Speaking from personal experience, I once consulted for a global service provider where we implemented this exact technique. Our policy contained four local-preference options and the first step is to configure the community-lists. These are the community matching constructs used in Cisco IOS. Each list will match exactly one community value, and these are arbitrary; they can be any value you prefer. Many service providers will use a format where the first value is their local AS, which is 65345 in our demo. The second value is often the desired local-preference value, making the community value self-documenting. Service providers typically publish these communities on their support portals for quick reference; customers don’t even need to ask permission before leveraging them!

# R4
ip community-list standard CL_LPREF_80 permit 65345:80
ip community-list standard CL_LPREF_90 permit 65345:90
ip community-list standard CL_LPREF_110 permit 65345:110
ip community-list standard CL_LPREF_120 permit 65345:120

Here’s a Cisco IOS example route-map that illustrates the conditional logic. Assuming a default local-preference of 100, customers can choose from values of 80, 90, 110 or 120. Therefore, customers have two options to make a prefix less preferred (80 and 90) and two options to make a prefix more preferred (110 and 120). Note that the policy should have a catch-all clause that handles routes without any communities attached. In this example, I’m permitting all routes with no modification, but this default action may vary between environments. R4 will apply this policy inbound on the BGP session with R1.

# R4
route-map RM_CUSTOMER_COMM_LPREF_IN permit 10
 match community CL_LPREF_80
 set local-preference 80
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 20
 match community CL_LPREF_90
 set local-preference 90
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 30
 match community CL_LPREF_110
 set local-preference 110
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 40
 match community CL_LPREF_120
 set local-preference 120
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 1000
!
router bgp 65345
 address-family ipv6
  neighbor 2001:DB8:1:4::1 route-map RM_CUSTOMER_COMM_LPREF_IN in

The final step is to enable the transmission of communities from R1 to R4, at a minimum. Cisco IOS does not send communities to BGP peers by default. This step also includes applying the desired community value outbound. To keep this blog and the subsequent configurations as simple as possible, I’m not matching specific prefixes for this policy application. Feel free to apply more precise matching criteria in your production environments as necessary. R1 must apply either 65345:80 or 65345:90 in this design, signaling that AS 65345 should not prefer R4 as an egress point towards R1.

# R1
route-map RM_BGP_COMM_R4_OUT permit 10
 set community 65345:80
!
router bgp 65001
 address-family ipv6
  network 2001:DB8::1/128
  neighbor 2001:DB8:1:4::4 send-community
  neighbor 2001:DB8:1:4::4 route-map RM_BGP_COMM_R4_OUT out

Now, let’s verify everything. On R4, let’s check the BGP table for 2001:db8::1/128, the prefix from R1. R4 has two paths. The eBGP path from R1 has a community of 65345:80, a local-preference of 80, and it is not the best path. The iBGP path from R2 has no community, a local-preference of 100 (default) and it is the best path. This iBGP path from R2 even has a giant AS-path, which R1 successfully overrode by signaling to AS 65345 that the egress point of R4 should be less preferred. The most important part of this output is the “localpref 80” text which indicates a successful policy implementation.

R4#show bgp ipv6 unicast 2001:db8::1/128
BGP routing table entry for 2001:DB8::1/128, version 21
Paths: (2 available, best #1, table default)
  Advertised to update-groups:
     4
  Refresh Epoch 6
  65002 65002 65002 65002 65001
    2001:DB8::3 (metric 10) from 2001:DB8::3 (192.0.2.3)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0
  Refresh Epoch 8
  65001
    2001:DB8:1:4::1 (FE80::1) from 2001:DB8:1:4::1 (192.0.2.1)
      Origin IGP, metric 0, localpref 80, valid, external
      Community: 65345:80
      rx pathid: 0, tx pathid: 0

Since R4 learned its best path from R3, let’s head there next. There’s only one BGP route available, which is the best path via R2 using eBGP. Again, the giant AS-path doesn’t matter since the local-preference set by R4 is less than the local-preference set by R3. BGP routers can only advertise their best paths, so none of the other AS 65345 routers currently know about R4’s backup path. Designing a BGP fast-failover backup plan is a problem for another day.

R3#show bgp ipv6 unicast 2001:db8::1/128
BGP routing table entry for 2001:DB8::1/128, version 27
Paths: (1 available, best #1, table default)
  Advertised to update-groups:
     3
  Refresh Epoch 5
  65002 65002 65002 65002 65001
    2001:DB8:2:3::2 (FE80::2) from 2001:DB8:2:3::2 (192.0.2.2)
      Origin IGP, localpref 100, valid, external, best
      rx pathid: 0, tx pathid: 0x0

We can continue our network verification by heading to R5. R5 is only aware of the egress path via R3 using iBGP. Again, without using the community/local-preference technique just described, R1 would be left performing trial-and-error AS-path prepending (not ideal).

R5#show bgp ipv6 unicast 2001:db8::1/128
BGP routing table entry for 2001:DB8::1/128, version 20
Paths: (1 available, best #1, table default)
  Advertised to update-groups:
     2
  Refresh Epoch 6
  65002 65002 65002 65002 65001
    2001:DB8::3 (metric 10) from 2001:DB8::3 (192.0.2.3)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0

Let’s quickly confirm that traffic from R6, a device beyond AS 65345, can reach R1 via R2 instead of the direct R1-R4 link. Using traceroute, we target R1’s loopback sourced from R6’s loopback, and the routed path is R5-R3-R2-R1 as expected.

R6#traceroute 2001:db8::1 source 2001:db8::6
Type escape sequence to abort.
Tracing the route to 2001:DB8::1

  1 2001:DB8:5:6::5 6 msec 5 msec 5 msec
  2 2001:DB8::3 4 msec 6 msec 5 msec
  3 2001:DB8:2:3::2 2 msec 6 msec 5 msec
  4 2001:DB8:1:2::1 2 msec 2 msec 5 msec

To summarize: If you find yourself wanting to influence ingress traffic towards your AS but the traditional methods of MED, AS-path prepending, or longest-match routing are inadequate or unavailable to you, check your service provider’s website. You might be pleasantly surprised! If you enjoyed this blog and my no-nonsense, tech-focused style of training, you’ll love my Cisco Advanced Routing series on Pluralsight.

Reference configurations

hostname R1
ipv6 unicast-routing
ipv6 cef
!
interface Loopback0
 ipv6 address 2001:DB8::1/128
!
interface Ethernet0/0
 ipv6 address FE80::1 link-local
 ipv6 address 2001:DB8:1:4::1/64
!
interface Ethernet0/1
 ipv6 address FE80::1 link-local
 ipv6 address 2001:DB8:1:2::1/64
!
router bgp 65001
 bgp router-id 192.0.2.1
 no bgp default ipv4-unicast
 neighbor 2001:DB8:1:2::2 remote-as 65002
 neighbor 2001:DB8:1:4::4 remote-as 65345
 !
 address-family ipv6
  network 2001:DB8::1/128
  neighbor 2001:DB8:1:2::2 activate
  neighbor 2001:DB8:1:2::2 route-map RM_BGP_LPREF_R2_IN in
  neighbor 2001:DB8:1:4::4 activate
  neighbor 2001:DB8:1:4::4 send-community
  neighbor 2001:DB8:1:4::4 route-map RM_BGP_COMM_R4_OUT out
 exit-address-family
!
ip bgp-community new-format
!
route-map RM_BGP_LPREF_R2_IN permit 10
 set local-preference 200
!
route-map RM_BGP_COMM_R4_OUT permit 10
 set community 65345:80

hostname R2
ipv6 unicast-routing
ipv6 cef
!
interface Ethernet0/0
 ipv6 address FE80::2 link-local
 ipv6 address 2001:DB8:2:3::2/64
!
interface Ethernet0/1
 ipv6 address FE80::2 link-local
 ipv6 address 2001:DB8:1:2::2/64
!
router bgp 65002
 bgp router-id 192.0.2.2
 no bgp default ipv4-unicast
 neighbor 2001:DB8:1:2::1 remote-as 65001
 neighbor 2001:DB8:2:3::3 remote-as 65345
 !
 address-family ipv6
  neighbor 2001:DB8:1:2::1 activate
  neighbor 2001:DB8:2:3::3 activate
  neighbor 2001:DB8:2:3::3 route-map RM_BGP_ASPREP_R3_OUT out
 exit-address-family
!
ip bgp-community new-format
!
route-map RM_BGP_ASPREP_R3_OUT permit 10
 set as-path prepend 65002 65002 65002

hostname R3
ipv6 unicast-routing
ipv6 cef
!
interface Loopback0
 ipv6 address 2001:DB8::3/128
 ospfv3 1 ipv6 area 0
!
interface Ethernet0/0
 ipv6 address FE80::3 link-local
 ipv6 address 2001:DB8:2:3::3/64
!
interface Ethernet0/1
 ipv6 address FE80::3 link-local
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
interface Ethernet0/2
 ipv6 address FE80::3 link-local
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
router ospfv3 1
 router-id 192.0.2.3
 !
 address-family ipv6 unicast
 exit-address-family
!
router bgp 65345
 bgp router-id 192.0.2.3
 no bgp default ipv4-unicast
 neighbor 2001:DB8::4 remote-as 65345
 neighbor 2001:DB8::4 update-source Loopback0
 neighbor 2001:DB8::5 remote-as 65345
 neighbor 2001:DB8::5 update-source Loopback0
 neighbor 2001:DB8:2:3::2 remote-as 65002
 !
 address-family ipv6
  neighbor 2001:DB8::4 activate
  neighbor 2001:DB8::4 next-hop-self
  neighbor 2001:DB8::5 activate
  neighbor 2001:DB8::5 next-hop-self
  neighbor 2001:DB8:2:3::2 activate
 exit-address-family
!
ip bgp-community new-format


hostname R4
ip cef
ipv6 unicast-routing
ipv6 cef
!
interface Loopback0
 ipv6 address 2001:DB8::4/128
 ospfv3 1 ipv6 area 0
!
interface Ethernet0/0
 ipv6 address FE80::4 link-local
 ipv6 address 2001:DB8:1:4::4/64
!
interface Ethernet0/1
 ipv6 address FE80::4 link-local
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
interface Ethernet0/3
 ipv6 address FE80::4 link-local
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
router ospfv3 1
 router-id 192.0.2.4
 !
 address-family ipv6 unicast
 exit-address-family
!
router bgp 65345
 bgp router-id 192.0.2.4
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 2001:DB8::3 remote-as 65345
 neighbor 2001:DB8::3 update-source Loopback0
 neighbor 2001:DB8::5 remote-as 65345
 neighbor 2001:DB8::5 update-source Loopback0
 neighbor 2001:DB8:1:4::1 remote-as 65001
 !
 address-family ipv6
  neighbor 2001:DB8::3 activate
  neighbor 2001:DB8::3 next-hop-self
  neighbor 2001:DB8::5 activate
  neighbor 2001:DB8::5 next-hop-self
  neighbor 2001:DB8:1:4::1 activate
  neighbor 2001:DB8:1:4::1 route-map RM_CUSTOMER_COMM_LPREF_IN in
 exit-address-family
!
ip bgp-community new-format
ip community-list standard CL_LPREF_80 permit 65345:80
ip community-list standard CL_LPREF_90 permit 65345:90
ip community-list standard CL_LPREF_110 permit 65345:110
ip community-list standard CL_LPREF_120 permit 65345:120
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 10
 match community CL_LPREF_80
 set local-preference 80
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 20
 match community CL_LPREF_90
 set local-preference 90
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 30
 match community CL_LPREF_110
 set local-preference 110
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 40
 match community CL_LPREF_120
 set local-preference 120
!
route-map RM_CUSTOMER_COMM_LPREF_IN permit 1000

hostname R5
ipv6 unicast-routing
ipv6 cef
!
interface Loopback0
 ipv6 address 2001:DB8::5/128
 ospfv3 1 ipv6 area 0
!
interface Ethernet0/0
 ipv6 address FE80::5 link-local
 ipv6 address 2001:DB8:5:6::5/64
!
interface Ethernet0/2
 ipv6 address FE80::5 link-local
 ipv6 address 2001:DB8:3:5::5/64
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
interface Ethernet0/3
 ipv6 address FE80::5 link-local
 ospfv3 1 ipv6 area 0
 ospfv3 1 ipv6 network point-to-point
!
router ospfv3 1
 router-id 192.0.2.5
 !
 address-family ipv6 unicast
 exit-address-family
!
router bgp 65345
 bgp router-id 192.0.2.5
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 2001:DB8::3 remote-as 65345
 neighbor 2001:DB8::3 update-source Loopback0
 neighbor 2001:DB8::4 remote-as 65345
 neighbor 2001:DB8::4 update-source Loopback0
 neighbor 2001:DB8:5:6::6 remote-as 65006
 !
 address-family ipv6
  neighbor 2001:DB8::3 activate
  neighbor 2001:DB8::3 next-hop-self
  neighbor 2001:DB8::4 activate
  neighbor 2001:DB8::4 next-hop-self
  neighbor 2001:DB8:5:6::6 activate
 exit-address-family
!
ip bgp-community new-format



Related tags:

it-ops   networking   bgp  
About the author

Nicholas (Nick) Russo, CCDE #20160041 and CCIE #42518, is an internationally recognized expert in IP/MPLS networking and design. To grow his skillset, Nick has been focused advancing Network DevOps via automation for his clients. Recently, Nick has been sharing his knowledge through online video training and speaking at industry conferences. Nick also holds a Bachelor’s of Science in Computer Science from the Rochester Institute of Technology (RIT). Nick lives in Maryland, USA with his wife, Carla, and daughter, Olivia.

10-day free trial

Sign Up Now