Monitoring BGP Routes with ThousandEyes

Posted by on March 5, 2014

In this post I’ll cover in more detail the route monitoring capabilities of ThousandEyes, already touched in previous blog posts on how routing changes impact performance, and BGP for DDoS mitigation. Routing is a key determinant of network performance; each route that packets take has varying latencies and throughput. And when routing goes wrong, it can prevent packets from getting to their destination. Therefore, understanding routing across networks, specifically using Border Gateway Protocol (BGP), is critical to troubleshooting traffic flows that traverse large corporate networks or the public Internet.

A Brief Intro to the Border Gateway Protocol (BGP)

The Internet consists of a myriad of independent networks organized into Autonomous Systems (AS). Each AS typically represents an independent administrative domain managed by a single organization and identified by a 4-byte number, e.g. AS 7018 is AT&T, AS 701-703 is Verizon, etc. Inside each AS there are a series of border routers (e.g. 2a-2c in Figure 1) that typically connect to each other in a full mesh using iBGP (i=internal; reflectors and confederations can be used to relax this constraint). Border routers in different ASes connect to each other through eBGP (e=external) sessions. BGP is used to announce reachability to a chunk of IP addresses (or prefix). BGP defines more than just physical interconnections; it is used to advertise which routes are possible based on policies defined by other considerations such as traffic engineering, maintenance, and commercial transit and peering agreements.

For example, pinterest.com resolves to the IP address 23.23.131.240. If we look at routing tables for announced address blocks that cover this IP address, we find it falls under address block 23.22.0.0/15 announced by AS 14618 belonging to Amazon.

Looking at RouteViews route server (telnet://route-views.routeviews.org) we can check the different AS paths available to reach 23.22.0.0/15. In the case below there are 31 available routes to reach the destination, but the router only picks one, the BGP best path, which is selected after looking at several route attributes, including BGP Local Preference and AS Path length.

eBGP sessions glue different ASes together, and iBGP sessions connect routers within the same AS.
Figure 1: eBGP sessions glue different ASes together, and iBGP sessions connect routers within the same AS.

External BGP Visibility (outside-in)

Public sources of BGP data, including RIPE-RIS in Europe and RouteViews in the U.S. establish eBGP sessions with hundreds of routers across the world (monitors) and provide a comprehensive picture of global routing reachability for a certain prefix (outside-in). This is the picture ThousandEyes typically represents in our BGP Route Visualization. For instance, in Figure 2 AS 36175 (ancestry.com) is announcing prefix 66.43.20.0/22 to 2 upstream providers XO Communications (AS 2828) and American Fiber (AS 31993). Each of the small green circles represent a router (or monitor) that is proving public BGP feeds. In the timeline, we are representing the average number of path changes per monitor; other metrics such as reachability and number of updates are also available. In this case, we noticed there’s a bump in the number of path changes at 6:00 UTC. If we zoom into that instant of time (Figure 3), we can see that there was a route change from AS 2828 (XO) to AS 31993 (American Fiber).

Ancestry.com (AS 36175) route visualization.
Figure 2: Ancestry.com (AS 36175) route visualization.
A routing change between different providers caused packet loss across the network for AS 36175.
Figure 3: A routing change between different providers caused packet loss across the network for AS 36175.

Internal BGP Visibility (inside-out)

We recently released the capability of visualizing both public and private eBGP routes for our customers. This means that any of our customers can setup a multi-hop eBGP session between one of their BGP speakers and our route collectors. There are two main benefits:

  • Internal prefixes: for prefixes originated inside the network, the private feed is useful to triage problems whose root cause is inside the network versus problems that originate outside; users will be provided with a single view of public and private feeds.
  • External prefixes: for prefixes belonging to a third party (e.g. a SaaS provider), the private feed is useful to detect cases where the route to the destination is sub-optimal (which affects performance of the application), or the route is taking a detour to a malicious destination (route hijacking).

Figure 4 below shows an example of one of our customers’ internal prefixes as seen by a combination of public and private BGP monitors. The small green double circle is a private BGP monitor. We can see that there are two origin Autonomous Systems in this view (the big green circles in the middle), but private AS 64999 in this case is only seen by the private monitor, and it’s not exposed to the other monitors.

Monitoring iBGP routes
Figure 4: Integrating private and public BGP feeds into a single view.

Setting Up Private BGP Feeds in ThousandEyes

Setting up a private BGP feed with us is pretty straightforward. You just need to go to “Settings -> My Domains & Networks -> Private BGP Monitors” and complete the form indicating your router IP address and ASN, and we will coordinate with you to bring the session up (Figure 5). You can check the status of your sessions in the table at the bottom of this page as well.

ThousandEyes BGP tests
Figure 5: Configuring a new private BGP session with ThousandEyes.

With the combination of both public and private eBGP visibility, ThousandEyes provides a greater understanding of routing issues that occur within a corporate network as well as issues with external prefixes. This information can help reduce latencies, spot inefficient routes, troubleshoot incorrect routing changes, and detect hijacked routes. Start monitoring BGP routes in ThousandEyes by signing up for a free trial.

Processing...
  • Juan

    Can i use this features with lite version?

    • Nick Kephart

      Yes, you can use BGP Private Peers with a Lite account. You’ll need to have an Enterprise Agent active in your account so you can create a Network or Web test; then add your private peer to the list of BGP monitors in Advanced Settings.