AWS Latency Index

in modern web applications, it is of paramount importance to deal with latency correctly. within an application, latency reveals itself at many different levels and with very different values. computer latency at a human level is a good reference to understand the different timescales.

it is clear that the dominant term, by large, is the network latency. this is something to bear in mind especially when using a microservices architecture.

however, very few of us are familiar with the latency value of an internet call between two locations on earth.

this post aims to shed some light on the subject by presenting some data collected on the aws infrastructure.

so, in the same way that the ubiquity of mcdonald's around the world has allowed economists to create the big mac index , the ubiquity of aws regions across the globe will let us create the aws latency index to measure the performance of internet calls.

test description and results

the test consists of four web servers in four different aws regions: ireland, oregon, sydney, and sao paulo, pinging each other once every minute. the servers return an empty response (http code 204). the latency of each call is stored in a database for later analysis.

the test was run in two different periods: from 2018-09-28 to 2018-10-23 and from 2019-02-23 to 2019-03-31.

here’s the average value of the latency (in seconds) for each period:

a first look at the values reveals that, in most cases, the results are similar in both directions up to the order of units of milliseconds. this is good news; it would have been worrying, had the results been different. from now on, we will consider just one of the directions for each par of regions.

the second thing to notice is that the values of the second period are consistently better, for instance, the improvement between oregon and sydney is about 31 ms and between sao paulo and sydney is a whopping 43 ms.

to make it clearer, let’s present the data of each direction just for the second period:

the values vary from 267 ms between oregon and ireland up to 630 ms between sao paulo and sydney. according to computer latency at a human level , that is equivalent to 21 and 49 years, respectively. just an eternity!

the following histogram provides a visual representation of the latency between oregon and the other regions:

it is interesting to compare the above graph with the one corresponding to the first period:

oregon latency – october 2018

not only is the average latency better in the second period, but the values are more concentrated around the average value, making a nice distribution graph.

here are the graphs corresponding to the other regions:

if instead of histograms we use time series analysis, we can get some interesting insights about the data. the below graph represents the values of latency between ireland and oregon during the sampled period (values are resampled into bins of 1 hour). the values over the 95th percentile are few and not very large, which is consistent with the histogram representation.

if we compare the above graph to the one corresponding to the first period, the difference is noticeable. the number of outliers is higher and their values greater.

after removing the values above the 95th percentile, the above graphs become:

the values of march are very well-behaved, with the median and the mean having nearly the same values. the same cannot be said of the values corresponding to october.

latency index

let’s focus on the results of march to calculate the latency index. we will define the latency index as the ratio between the theoretical value of the latency and the real one.

the theoretical value corresponds to the time it would take to cover the distance between two regions and back along the great circle connecting those regions and travelling at the speed of light.

the values vary from nearly 30 percent between oregon and sydney to just 14 percent between sao paulo and sydney.

conclusion

the latency measurements obtained during the period of march 2019 are better behaved than those corresponding to october 2018. this may be down to either problems on aws infrastructure back in october or problems on the internet connectivity itself.

as to the latency index, it is a bit disappointing. the best result, for oregon-sydney, is just 30 percent. in other words, in the best case scenario, the latency is more than 3 times the theoretical value.

in the case of sao paulo-sydney, it is 7 times! most likely, there is not a direct cable between south america and australia or the internet infrastructure in south america is not that great.

the application and the data collected can be found on the aws index . that repository also contains two jupyter notebooks written in python with all the details of the data analysis and calculations shown in this post. the notebooks can be executed online on https://nbviewer.jupyter.org/.