The Networks Behind the 2018 Double 11 Global Shopping Festival
Learn how Alibaba Cloud handles network traffic on their busiest shopping day of the year.
Join the DZone community and get the full member experience.Join For Free
The Alibaba Cloud Double 11 Shopping Festival. At the stroke of midnight, when the rolling counter stopped, 2018 GMV was RMB213.5 billion ($30.8 billion), 27% higher than last year's RMB168.3 billion. As Double 11 wore on, milestones from the previous 10 years fell, one by one. Just after 4:00 p.m., it was a foregone conclusion that this year would be the biggest Double 11 ever – something Alibaba Group executives had promised heading into the shopping festival. The success of Double 11 relies heavily on sophisticated and massive network services. This include network services such as Cloud Enterprise Network (CEN), Virtual Private Cloud (VPC), Server Load Balancer (SLB), network address translation (NAT) gateway, and Express Connect.
How exactly do these network services and technologies support the Double 11 Global Shopping Festival? How can they affect or improve our shopping experience? This article seeks to answer these questions by showing you the various network services and technologies used for the 2018 Double 11 Shopping Festival.
VPC – A Secure Network Container
VPC is a secure network environment configured with individual "private network containers" to logically isolate users in the cloud. Those containers can store user-required cloud services and resources, such as Server Load Balancer and RDS.
VPC is the basis for assigning network management capabilities to users in the cloud, such as IP address range selection, subnet division, gateway configuration, private network intercommunication, and IDC intercommunication off the cloud. VPC allows users to control their networks.
As a platform shared by many users, the public cloud platform carries the core transactions, orders, and logistics of e-commerce during the Double 11 Global Shopping Festival. Tenants are isolated by VPCs to ensure transaction security during the Double 11 Global Shopping Festival.
As shown in the following figure, a VPC on the public cloud platform is used during the Double 11 Global Shopping Festival. The VPC is isolated from other VPCs and communication is disabled between this VPC and other VPCs.
VPC is more secure than a classic network because the former uses the tunneling technology for logical isolation. The tunneling technology adopted by VPC is like opening a dedicated tunnel for each user to pass along a highway, with every tunnel completely isolated from the others. The secure isolation technology adopted by a classic network is like dividing different lanes on a highway, with every lane isolated from the others by a buffer zone. The tunneling technology is more secure than the secure isolation technology.
As shown in the following figure, Tunnel 100 and Tunnel 200 correspond to the VPCs of two different users. One VPC contains VM1 and VM3, and the other contains VM2 and VM4. The communication of each VM is carried by a dedicated tunnel, and each VM is isolated from the VMs in other tunnels.
You may wonder if a VPC has enough capacity to store a large number of ECS instances. Some users on the public cloud platform store more than 20,000 ECS instances in a single VPC. In a container environment, a single VPC can store more than 50,000 containers, which makes it easy to satisfy the resource requirements of services.
Server Load Balancer – Reducing Traffic Peaks
Server Load Balancer distributes traffic among multiple ECS instances to improve the serviceability of application systems. Server Load Balancer has been the portal of critical service systems. A large number of ECS instances are required to handle the traffic peaks generated by the access of hundreds of millions of users during the Double 11 Global Shopping Festival. The ECS instances are scheduled by Server Load Balancer, which receives user requests, allocates them intelligently to backend ECS instances for handling, and returns results to users. See the following figure.
Server Load Balancer plays a vital role for handling traffic peaks and ensuring smooth user communication experience during the Double 11 Global Shopping Festival. Processing capability and scheduling efficiency are essential for Server Load Balancer and can be measured by the following technical indicators:
- Peak traffic per second
- New connections per second (CPS)
- Maximum concurrent connections (MaxConn)
- Queries per second (QPS)
- Schedulable backend ECS instances
The following is data of an instance to indicate the performance of Server Load Balancer during the Double 11 Global Shopping Festival.
The data is sourced from only one instance. Server Load Balancer uses many instances during the Double 11 Global Shopping Festival. The processing capability of Server Load Balancer clusters varies depending on different cluster sizes. A single cluster in the public cloud provides more than 100 million maximum connections, more than 10 million new connections per second, and total 640 Gbps bandwidth.
What is the configuration of the Server Load Balancer instances used by the Double 11 Global Shopping Festival? How are Server Load Balancer clusters optimized? If you are interested, see the Unveiling the Networks Behind the Double 11 Global Shopping Festival in 2017 – An Insight into Server Load Balancer Handling Traffic Peaks During the Events
NAT Gateway – Essential for Successful Payments
Alibaba Cloud NAT gateway supports the SNAT and DNAT functions. SNAT allows the ECS instances in a VPC to access the Internet without a public network connection. DNAT maps public IP addresses to the ECS instances in a VPC so that the ECS instances provide services on the Internet.
During the Double 11 Global Shopping Festival, the NAT gateway mainly provides the SNAT service and plays a vital role in successful payment. As shown in the following figure, a user selects an item and clicks Submit Order.
The system jumps to Alipay and the NAT gateway implements SNAT to call the payment interface of Alipay so that the user makes a payment, as shown in the following figure.
The NAT gateway features ultra-large bandwidth, massive concurrent connections, and a robust disaster tolerance capability, enabling Alipay to handle a peak payment amount of 256,000 payments per second during the Double 11 Global Shopping Festival. One of the NAT gateways supporting the Double 11 Global Shopping Festival handled 3 million maximum connections.
Express Connect – The World's Largest Network Channel in Hybrid Cloud
The hybrid cloud enables intercommunication between the public cloud and IDCs off the cloud. It not only protects offline IDC investments but also fully utilizes cloud elasticity, which is particularly suitable for the Double 11 Global Shopping Festival. After nine years of development, the Double 11 Global Shopping Festival goes beyond the consumer and retail sector, boosts unprecedented social collaboration, and has become the largest laboratory for commercial, economic, and technical transformations. Therefore, the Double 11 Global Shopping Festival is the best practice of the largest hybrid cloud architecture in the world.
The hybrid cloud carries the traffic of item browsing, order payment, customer service, and logistics query, during which many systems are called frequently between public cloud and IDCs off the cloud, based on Express Connect.
As shown in the following figure, Express Connect connects off-the-cloud IDCs and VPC in the cloud by using leased lines and connects different VPCs across regions.
Luoshen – The System Behind Network Services
Apart from network services, networks are the infrastructure of other systems or services during the Double 11 Global Shopping Festival. For example, the database used by the events reaches a peak processing capacity of 42 million times per second, thanks to the underlying network technology. Another example is VSwitch residing on a host machine, used to ensure the live streaming of the Double 11 Global Shopping Festival evening party and the network performance of ECS instances. The network is the basis for the hundreds of thousands of systems during the Double 11 Global Shopping Festival. What is the virtual network system behind the Double 11 Global Shopping Festival? The following describes Luoshen, a virtual network system of Apsara briefly. It will be described in detail later.
Luoshen is divided into the control plane, data plane, and operation plane. The control plane uses SDN to manage and control networks. For example, VPC ECS instances are activated at the control plane. The data plane mainly forwards data. For example, its modules allow EC2 to access the Internet or other ECS instances or ensure the network performance of ECS instances. QITAN is an intelligent network analysis platform which implements intelligent analysis and real-time monitoring of the entire virtual network system. It has intelligent robots to provide services.
Published at DZone with permission of Leona Zhang. See the original article here.
Opinions expressed by DZone contributors are their own.