Now, more than ever, the Internet has become a critical lifeline for billions of people around the world impacted by COVID-19. A vast portion of the workforce is working remotely. Students at every level are learning online. Fitness classes, religious services, and medical appointments have gone digital. Social interactions are now mostly virtual. Consumer networks have effectively become our all-in-one tether to the world, connecting every one of us to critical services and each other — and this macro traffic shift has not gone unnoticed. Verizon has reported a 20% increase in overall traffic on its network in just the last week. Vodafone has reported a 50% increase.
So, how is the Internet holding up under the weight of this unprecedented (and unforeseen) level of usage? So far, reasonably well.
Despite massive traffic increases — particularly across consumer last-mile networks — we have not seen a significant corresponding spike in Internet outages, which can occur when traffic levels strain network capacity. However, there has been an upward trend line in outages over the last three weeks compared with the previous three-week baseline. We have also observed some performance degradation across key UCaaS services.
To illustrate the current state of the Internet, we’re going to explore its health from three angles: ISPs, public cloud, and UCaaS performance.
Internet Service Providers
Looking at ISP outages within the last six weeks, we can see that the number of outages increased globally. Sporadic increases are not atypical under normal Internet conditions, as the number of daily and weekly outages can change for a variety of (mostly unpredictable) reasons. However, we have seen a concerning upward trajectory since the beginning of this month coinciding with the broader spread of COVID-19 and subsequent changes in Internet usage.
ISP outages rose even more dramatically in the United States, with an almost 100% increase between early February and the beginning of March. That early March level has been mostly sustained over the last couple of weeks.
While we’ve seen an increase in outages overall, two transit providers, in particular, Cogent Communications (AS 174) and Hurricane Electric (AS 6939) had very large-scale outages over this period.
On March 11th, Cogent experienced service disruptions for the fairly lengthy period (by Internet standards) of 30 minutes.
One week later, on March 18th, Cogent suffered a nearly identical outage in scope and length that impacted its ISP peers, as well as the reachability of services such as Amazon and LinkedIn.
Yet another notable outage occurred just after midnight last Friday, March 20th. The outage within Hurricane Electric’s network wasn’t as far-reaching or lengthy as either Cogent outage, but it was accompanied by smaller, intermittent disruptions that collectively impacted hundreds of sites and services.
Public Cloud Provider Networks
Major public cloud providers, such as AWS, Microsoft Azure, and Google Cloud, have built massive global networks that are incredibly well-equipped to handle traffic surges. So, unsurprisingly, we have seen almost no COVID-19 related impacts within their networks. Looking at the number of detected outages over the last six weeks, the trend line remains relatively level globally, with only a slight increase within the United States.
As with ISPs, cloud network outages are generally unpredictable, and, when large-scale in nature, are often due to a routing or infrastructure state change, rather than traffic congestion. Luckily for its enterprise customers and their users, cloud performance has held steady in the last couple of weeks.
Reachability and Performance of Top UCaaS Providers
Collaboration applications have seen a dramatic rise in usage over the last several weeks. Looking at the global performance of these services between March 9–20 compared to the previous two-week time span, we can see that the top three experienced relative performance impacts. Interestingly, one of the providers actually showed improvements in availability, latency, reduction in packet loss and jitter, while the other two showed minimal (in the grand scheme of things) degradations on all fronts — not surprising given the unprecedented strain they’ve been under.
There were sporadic increases in traffic loss for users connecting to these services over the last two weeks, as seen in Figure 9.
Traffic loss could be caused by services and factors external to the provider, for example, if ISP networks neighboring these providers experience congestion or infrastructure issues. However, in the case of one of the providers, outages within its own network spiked last week, meaning that the network issues impacting users were taking place on infrastructure managed by the provider versus an external ISP. The outage incidents that took place this past week contributed to the 4% decline in availability of its service.
Outage incidents within large UCaaS provider networks are fairly infrequent; however, the recent massive surge in usage is clearly stressing current design limits. Capacity is reportedly being added across the board to meet new service demands.
Not all performance issues we observed have been related to usage volume. For example, this past Friday, March 20th, many users were unable to reach one of the provider’s services because some of its domains were not resolving to an IP address.
The disruption lasted approximately 20 minutes, during which time, its service was still operational, but simply not findable due to a DNS resolution failure. The issue appears to be related to a DNS configuration change made around that time. Once the misconfiguration was corrected, the reachability of its service resumed.
UCaaS applications will likely continue to see users and usage levels increase in the coming weeks and months, so we will continue to track and provide regular updates on their performance.
Introducing the ThousandEyes Global Internet Outages Map
In an effort to provide ongoing insight into the state of the global Internet, today we announced a publicly-available Global Internet Outages Map showing ongoing and recent network outages.
The map is presented in near real time (updated every few minutes), so be sure to bookmark the page to stay updated on outages taking place in ISP, public cloud, UCaaS and edge service provider networks.
Moving Into the Internet Unknown
While — so far — the Internet is not buckling under the weight of huge traffic increases, it’s early days and no one can predict what will happen next. We are actively monitoring global Internet performance and, in addition to our Internet Map, we will be providing regular updates and analysis to help you continue to understand the state of the Internet under COVID-19.