Contact us
April 6th, 2020

Weekend route leak by AS7552

On Sunday of April 5, 2020, only a few days after last week route leaks an AS7552 belonging to Viettel - according to Wikipedia the largest telecommunication service provider in Vietnam - was leaking routes for more than 3 hours in a row.


The leak affected 4825 network prefixes from 326 operators, spreading from AS7552 upstreams: AS3491 and AS4637 towards AS1273 - Vodafone, which helped spread it to almost all major Tier-1 ISPs. Most of all Vietnamese, Cambodian and Australian networks were affected, with more than 25% of ISPs in the first two countries.

             country              | prefix_count | origin_count | origins_in_country 
----------------------------------+--------------+--------------+--------------------
 Australia                        |         1469 |          107 |               2630
 Viet Nam                         |         1932 |          104 |                365
 Cambodia                         |          892 |           35 |                128
 Hong Kong                        |           23 |           15 |                987
 United States                    |           49 |           14 |              27589
 Lao People's Democratic Republic |           64 |            9 |                 29
 India                            |           20 |            8 |               2574
 Taiwan, Province of China        |            8 |            7 |                336
 Thailand                         |            5 |            4 |                527
 New Caledonia                    |           12 |            3 |                 14
 Singapore                        |            4 |            3 |                554
 Timor-Leste                      |           28 |            2 |                 12
 New Zealand                      |            2 |            2 |                643
 Indonesia                        |            2 |            2 |               1755
 Bangladesh                       |            2 |            1 |                986
 China                            |            8 |            1 |               1715
 France                           |            3 |            1 |               1873
 Korea, Republic of               |            1 |            1 |                924
 Malaysia                         |            4 |            1 |                278
 Myanmar                          |            4 |            1 |                 99
 Nepal                            |            1 |            1 |                114
 Pakistan                         |            1 |            1 |                198
 Philippines                      |          239 |            1 |                454
 United Kingdom                   |            1 |            1 |               2822
 Vanuatu                          |            1 |            1 |                 13
 Brazil                           |            1 |            1 |               7691

Interestingly enough, this case proves what we always see - the provider believes the customer - and the leak goes all the way through to the core of the internet with the help of AS7552 current upstreams. During the incident investigation, we also wanted to check how many Tier-1 operators accept the route containing another Tier-1 on a not adjacent place in AS-PATH, and it turned out that all of them do, except for NTT and GTT. Every other Tier-1 accepted this leak through peering with Telia - AS1299. We thought that Telia would drop such a route because maintaining such a filter from customer direction is relatively easy. And yet, it didn’t.


This is a part of the data from the RIPE analyze tool that illustrates routing history where we see that one of the prefixes that had only been seen locally, however, at the time of the leak, it became available globally: https://stat.ripe.net/widget/routing-history#w.resource=118.107.68.0%2F23.


The BGPlay widget also highlights what exactly happened with this prefix during the incident.

As the quarantine measures continue almost across the globe, the pressure under which networks continue to work only rises. It is now only a matter of time when the next networking incident occurs because the human factor is actually helping to cope with the consequences of rapidly growing traffic amounts. Engineers would still need to make configuration changes, and yes, sometimes it could go wrong.

We could only, again, point to the necessity of monitoring networks in real-time, because even hour-long outages could cause severe damages and failure cascades for entire businesses.

×

LOGIN
×
Contact us
×
Type:
Email:
Subject:
Message:

Thank you for feedback!

We will contact you by provided email address.