Processing...
All posts in: "Outage Reports"

Understanding Internet Exchanges via the DE-CIX Outage

Posted by on April 13th, 2018
April 13th, 2018

Internet Exchanges (IX) play an important “efficiency” role in the practical matter of physically connecting the networks that comprise the Internet. IXs are facilities that provide space, power and connectivity to member networks—typically Internet Service Providers (ISPs), telecom, cable and mobile carriers, content providers, web enterprises and sophisticated enterprises such as large financial services companies. This infrastructure is used by these networks to arrange peerings exchange regional, national and/or international IP traffic. By being able to peer with many different networks in one facility, networks can cost-effectively gain lower-latency connections than if they had to reach those networks via multiple transit hops. The availability of multiple peerings also increases network and transit path resiliency. … Read More »

Amazon AWS Outage a Lesson in Managing Cloud First Risks

Posted by on March 2nd, 2018
March 6th, 2018

On Friday, March 2nd, 2018, a power outage hit the AWS-East Region (Ashburn), affecting hundreds of critical enterprise services like Atlassian, Slack and Twilio. Significant corporate websites and Amazon’s own service offerings were impacted as well. ThousandEyes operates a global set of software agents performing Internet-aware network monitoring. We were able to catch this outage in the act, which rippled very quickly outward towards other services. Even customers relying on Amazon private connectivity service known as AWS Direct connect were affected.… Read More »

How GitHub Successfully Mitigated a DDoS Attack

Posted by on March 1st, 2018
March 5th, 2018

On Wednesday, February 28th, 2018 at 9:15am Pacific Standard Time, GitHub, the popular web-based hosting service for software development, was a victim of a powerful DDoS attack that impacted its global user base of 20M. It was one of the largest DDoS attacks, with attack traffic peaking at 1.3Tbps. Concerned users, some who were in the midst of a code check-in, took to Twitter to express their feelings while simultaneously looking for answers. Unfortunately for GitHub, their status page was also affected, adding a bit of anxiety to the equation.… Read More »

A Tale of Two Trading Sites Told by Network Monitoring

Posted by on February 22nd, 2018
February 22nd, 2018

How can network monitoring help understand online trading website performance? In the first days of February, 2018 the stock market went on a bit of wild ride, with high levels of volatility amidst heavy trading sessions. On February 5th, the Dow skidded almost 1600 points before recovering slightly to a loss of 1175 points for the day—its worst daily point drop ever. Asian markets soon experienced their own swoon. Then on February 6, U.S. markets bounced back—regaining nearly 570 points, while the FTSE experienced its biggest drop since Brexit. As of February 9, the volatility continued.… Read More »

Top Internet and Cloud Outages of 2017

Posted by on January 17th, 2018
January 17th, 2018

Now that we have rung in the new year and settled into our daily routine, let’s take a few minutes to reflect on the state of the Internet and key outages that shook the world in 2017. Over the past few years, as we analyze many such outage events, we’ve noticed an undeniable trend that governs today’s cloud era: The Internet is the backbone of human communication. We start and end our day with it. Be it calling a ride to work or ordering dinner on the way back, the Internet has a part to play in our quality of life. Which is why when it breaks, it causes havoc.… Read More »

Comcast Suffers Outage Due to Significant Level 3 BGP Route Leak

Posted by on November 7th, 2017
November 8th, 2017

On Monday morning, November 6th 2017, from 9:45am-11:25am Pacific, Comcast suffered a nationwide outage sending several million users of the popular Internet Service Provider into a frenzy. Our analysis has revealed that Level 3 leaked over a thousand routes belonging to Comcast subsidiary networks and their customers. The immediate effect was that instead of traffic going through the Comcast backbone to reach these networks, traffic was forced to go via Level 3.… Read More »

What Happened When Marketo’s Domain Name Expired?

Posted by on July 26th, 2017
August 1st, 2017

On Tuesday, July 25th, 2017 at 4:25am Pacific, all services related to Marketo’s domain “marketo.com” began experiencing an outage. The outage continues to have impacts at the time of writing. Marketo is a marketing automation software widely used by companies around the world—when Marketo went down on Tuesday, forms on many companies’ websites broke completely. Many users were unable to access the Marketo platform or even email Marketo employees, since Marketo mail servers were also inaccessible.… Read More »

Rostelecom Route Leak Targets E-Commerce Services

Posted by on May 1st, 2017
May 1st, 2017

Last week Rostelecom, one of Russia’s largest, partially state-owned Internet service providers, leaked dozens of routes pertaining to IP addresses that belong to major financial services firms. This route leak redirected Internet traffic bound for these financial firms to Rostelcom, as some routers around the Internet decided that the new, leaked routes were preferable to the legitimate ones. … Read More »

AWS S3 Outage Likely Caused by Internal Network Issue

Posted by on February 28th, 2017
November 10th, 2017

On Tuesday, February 28th, 2017, the US-East-1 region of Amazon Web Services S3 saw a complete outage from 9:40am to 12:36pm PST. AWS S3 (Simple Storage Service) is a cloud object storage solution that many services rely on to store and retrieve files from anywhere on the web. In addition, many other AWS services that depend on S3 – Elastic Load Balancers, Redshift data warehouse, Relational Database Service and others – also had limited to no functionality. … Read More »

Top Internet Outages of 2016

Posted by on December 20th, 2016
January 18th, 2017

As we sip on warm holiday beverages and gradually wind down the year, it’s rather customary to reflect on the past and contemplate the future. Following traditions, we took a stroll down memory lane analyzing the state of the Internet and the “cloud”. In today’s blog post we discuss some of the most impactful outages of 2016, understand common trends and evaluate some of the key learnings.… Read More »