Cloud Architecture

This article describes the design and creation of Amazon Virtual Private Network (VPC) using the VPC Designer tool and Cloud Formation templates. It also provides details of VPC Components such as Subnets, Route tables, Security Groups, Internet Gateway, NAT Gateway, VPC endpoints, Network Interfaces, Network Access Control Lists (ACLs), and VPC Peering. Amazon Virtual Private Cloud (VPC) enables you to create your own dedicated, logically isolated virtual private network in your AWS account. This virtual network closely resembles a traditional network that you operate in your own data center (on-premises). It provides the ability to define and have full control over the virtual network environment, including security, connectivity, and resource deployment. VPC spans multiple availability zones in an AWS Region. An Amazon VPC is primarily focused on the following capabilities: Isolating your own AWS resources from other AWS Accounts. Protecting your AWS resources deployed in VPC from network Threats. Routing network traffic to and from your resources deployed in VPC. Illustration of a Virtual Private Cloud with Subnets. VPC Components The fundamental VPC components are: VPC CIDR Block Subnets Security Groups Network Access Control Lists (ACLs) Gateways (Internet, NAT) Route Table VPC CIDR Block VPCs and Subnets are associated with an IP address range that is a part of a Classless Inter-Domain Routing (CIDR) block, which will be used to allocate private IP addresses for the resources deployed in VPC. You can create a VPC by assigning either an IPv4 or IPv6 address. Subnets A subnet is associated with a CIDR block that is a subset of the '/16 CIDR block' of its VPC. A subnet must reside in a single availability zone. You can deploy AWS resources in a VPC after adding subnets to it. Security Groups A security group is considered the first-level defense and acts as a firewall for AWS resources deployed in a subnet to control incoming and outgoing traffic. Inbound rules control the incoming traffic to the instances, whereas outbound rules control the outbound traffic from the instances within the VPC. Network Access Control Lists (ACLs) An NACL is considered the second-level defense and acts as a firewall to the subnet that controls incoming and outgoing traffic. We can create rules to allow or deny network traffic to specific protocols through specific ports and specific IP address ranges. Network ACLs are stateless and have separate inbound and outbound rules. Gateways A Gateway connects a VPC to another network. The AWS resources residing in a VPC require connectivity outside of the AWS network, i.e., to the internet. A VPC must be attached to the Internet Gateway to access the Internet. NAT Gateway is used to enable the resources residing in a private subnet to connect to the Internet or other AWS services. Route Table A Route Table contains a set of rules, called routes, that determine where network traffic from a subnet or gateway needs to be directed. A Route Table is always associated with a subnet. VPC Peering A VPC peering connection is a networking connection between two VPCs that enables you to route traffic between them using private IPv4 addresses or IPv6 addresses. We can create a VPC peering connection between our own VPCs or a VPC in another AWS account. Instances in either VPC can communicate with each other as if they are within the same network. The VPCs can be in different Regions i.e., also known as an inter-Region VPC peering connection. Prerequisites We have provided a sample solution along with this article. To deploy this solution, we must have the following: An AWS account. AWS CLI with administrator permissions. Basic knowledge of CloudFormation templates. Design a VPC Using VPC Design Tool Amazon VPC supports both IPv4 and IPv6 addresses. Here, we will use IPv4 to design a VPC. When we create a VPC, we must specify an IPv4 address for the VPC. The acceptable address block will be between a '/16 netmask' (65,536 IP address) and a '/28 netmask' (16 IP address). AWS recommends that we specify the CIDR block from the private address ranges specified in RFC 1918. RFC 1918 range Example CIDR block 10.0.0.0 - 10.255.255.255 (10/8 prefix) 10.0.0.0/16 172.16.0.0 - 172.31.255.255 (172.16/12 prefix) 172.31.0.0/16 192.168.0.0 - 192.168.255.255 (192.168/16 prefix) 192.168.0.0/20 RFC 1918 Standard Let us discuss more IPv4 addresses. An IPv4 address is 32 bits. An IP Address is shown as four decimal numbers representing 4 bytes (each byte is 8 bits, i.e. 8 * 4 = 32 bits). Example of an IP CIDR Range : CIDR RANGE Explanation 10. 9 . 0 . 0 /16 8 + 8 CIDR / 16, first two digits will not change in this IP address range.10 . 9 . (0 - 255) . (0 - 255) = 256 * 256 = 65536 private IP addresses are available for usage. 10. 9 . 0 . 0 /18 8 + 8 +2 CIDR / 18, first two digits will not change in this IP address, and the third digit allows a range between 0 through 63.10 . 9 . (0 - 63) . 0 = 64 * 256 = 16384 private IP addresses are available for usage. 10. 9 . 0 . 0 /22 8 +8 +6 CIDR / 18, first two digits will not change in this IP address, and the third digit allows a range between 0 through 3.10 . 9 . (0 - 3) . 0 = 4 * 256 = 1024 private IP addresses are available for usage. 10. 9 . 0 . 0/24 8 + 8 + 8 CIDR / 24, first three digits will not change in this IP address.10 . 9 . 0 . 0 - 255 = 256 private IP addresses are available for usage. 10. 9 . 0. 0 /26 8+8+8+2 CIDR / 18, first three digits will not change in this IP address, and fourthdigit will allow a range between 0 through 63.10 . 9 . 0 . (0 - 63) = 64 private IP addresses are available for usage. 10. 9 . 0 . 0/32 8 + 8 + 8+ 8 CIDR / 32, all the four digits will not change in this IP address.10 . 9 . 0 . 0 = 1 private IP address is available for usage. Let us now design a VPC and its subnets with an IP CIDR range. Please ensure the VPC CIDR range we use does not overlap with existing IP address ranges currently in use in your organization. I am using a VPC designer tool to design subnets in a VPC. VPC CIDR Range: 10.9.0.0/22 There are 1024 IP addresses available to allocate to our subnets. Let us design two subnets in a VPC, one public and another private. This example takes three availability zones into consideration. We allocate 256 IP addresses for each private subnet using a 'CIDR /24' range. We then allocate 64 IP addresses for the first two public subnets using a 'CIDR /26' range. We then assign not 64, but 128 IP addresses to the last public subnet using a 'CIDR /25' range. This is possible as we would be left with 64 unused IP addresses, which we can assign to our third public subnet. Refer to the screenshot below to understand how the allocation works. VPC Creation Let us now create a VPC using the information provided above from the VPC designer tool and design the CIDR range for our public and private subnets using the CloudFormation template. YAML AWSTemplateFormatVersion: "2010-09-09" Description: This template deploys a VPC with a pair of public and private subnets spread across three Availability Zones. It deploys an internet gateway, defining routes to public subnets. It deploys three NAT gateways (one in each AZ), and routes them to private subnets. It also creates NACls for each Public and Private subnet. Parameters: pVPCCIDRBlock: Description: Please enter the IP range for this VPC Type: String pPublicSubnet1CIDR: Description: Please enter the IP range for the public subnet in the first Availability Zone Type: String pPublicSubnet2CIDR: Description: Please enter the IP range for the public subnet in the second Availability Zone Type: String pPublicSubnet3CIDR: Description: Please enter the IP range for the public subnet in the third Availability Zone Type: String pPrivateSubnet1CIDR: Description: Please enter the IP range for the private subnet in the first Availability Zone Type: String pPrivateSubnet2CIDR: Description: Please enter the IP range for the private subnet in the second Availability Zone Type: String pPrivateSubnet3CIDR: Description: Please enter the IP range for the private subnet in the third Availability Zone Type: String Resources: rVPC: Type: AWS::EC2::VPC Properties: CidrBlock: !Ref pVPCCIDRBlock EnableDnsSupport: true EnableDnsHostnames: true rPublicSubnet1: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPublicSubnet1CIDR VpcId: !Ref rVPC AvailailityZone: !Select [0, !GetAZs ''] MapPublicIpOnLaunch: true Tags: - Key: Name Value: "Public Subnet (AZ1)" rPublicSubnet2: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPublicSubnet2CIDR VpcId: !Ref rVPC AvailailityZone: !Select [1, !GetAZs ''] MapPublicIpOnLaunch: true Tags: - Key: Name Value: "Public Subnet (AZ2)" rPublicSubnet3: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPublicSubnet3CIDR VpcId: !Ref rVPC AvailailityZone: !Select [2, !GetAZs ''] MapPublicIpOnLaunch: true Tags: - Key: Name Value: "Public Subnet (AZ3)" rPrivateSubnet1: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPrivateSubnet1CIDR VpcId: !Ref rVPC AvailailityZone: !Select [0, !GetAZs ''] MapPublicIpOnLaunch: false Tags: - Key: Name Value: "Private Subnet (AZ1)" rPrivateSubnet2: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPrivateSubnet2CIDR VpcId: !Ref rVPC AvailailityZone: !Select [1, !GetAZs ''] MapPublicIpOnLaunch: false Tags: - Key: Name Value: "Private Subnet (AZ2)" rPrivateSubnet3: Type: AWS::EC2::Subnet Properties: CidrBlock: !Ref pPrivateSubnet3CIDR VpcId: !Ref rVPC AvailailityZone: !Select [2, !GetAZs ''] MapPublicIpOnLaunch: false Tags: - Key: Name Value: "Private Subnet (AZ3)" rPublicRouteTable: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Public Routes" rPublicSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPublicRouteTable SubnetId: !Ref rPublicSubnet1 rPublicSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPublicRouteTable SubnetId: !Ref rPublicSubnet2 rPublicSubnet3RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPublicRouteTable SubnetId: !Ref rPublicSubnet3 rInternetGateway: Type: AWS::EC2::InternetGateway Properties: Tags: - Key: Name Value: "Internet Gateway" rInternetGatewayAttachment: Type: AWS::EC2::VPCGatewayAttachment Properties: InternetGatewayId: !Ref rInternetGateway VpcId: !Ref rVPC rNatGateway1EIP: Type: AWS::EC2::EIP DependsOn: rInternetGatewayAttachment Properties: Domain: vpc rNatGateway2EIP: Type: AWS::EC2::EIP DependsOn: rInternetGatewayAttachment Properties: Domain: vpc rNatGateway3EIP: Type: AWS::EC2::EIP DependsOn: rInternetGatewayAttachment Properties: Domain: vpc rNatGateway1: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt rNatGateway1EIP.AllocationId SubnetId: !Ref rPublicSubnet1 rNatGateway2: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt rNatGateway2EIP.AllocationId SubnetId: !Ref rPublicSubnet2 rNatGateway3: Type: AWS::EC2::NatGateway Properties: AllocationId: !GetAtt rNatGateway3EIP.AllocationId SubnetId: !Ref rPublicSubnet3 rPrivateRouteTable1: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Private Routes (AZ1)" rPrivateRouteTable2: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Private Routes (AZ2)" rPrivateRouteTable3: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Private Routes (AZ3)" rPrivateSubnet1RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPrivateRouteTable1 SubnetId: !Ref rPrivateSubnet1 rPrivateSubnet2RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPrivateRouteTable2 SubnetId: !Ref rPrivateSubnet2 rPrivateSubnet3RouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: RouteTableId: !Ref rPrivateRouteTable3 SubnetId: !Ref rPrivateSubnet3 rPublicDefaultRoute: Type: AWS::EC2::Route Properties: RouteTableId: !Ref rPublicRouteTable DestinationCidrBlock: 0.0.0.0/0 GatewayId: !Ref rInternetGateway rPriaveteNATGatewayRoute1: Type: AWS::EC2::Route Properties: RouteTableId: !Ref rPrivateRouteTable1 NatGatewayId: rNatGateway1 DestinationCidrBlock: 0.0.0.0/0 rPriaveteNATGatewayRoute2: Type: AWS::EC2::Route Properties: RouteTableId: !Ref rPrivateRouteTable2 NatGatewayId: rNatGateway2 DestinationCidrBlock: 0.0.0.0/0 rPriaveteNATGatewayRoute3: Type: AWS::EC2::Route Properties: RouteTableId: !Ref rPrivateRouteTable3 NatGatewayId: rNatGateway2 DestinationCidrBlock: 0.0.0.0/0 rPublicSubnetNacl: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Public Subnet Nacl" rPrivateSubnetNacl: Type: AWS::EC2::NetworkAcl Properties: VpcId: !Ref rVPC Tags: - Key: Name Value: "Private Subnet Nacl" rPublicSubnetInbound: Type: AWS::EC2::NetworkAclEntry Properties: NetworkAclId: !Ref rPublicSubnetNacl Egress: false RuleNumber: 10 Protocol: 6 RuleAction: 'allow' CidrBlock: '0.0.0.0/0' PortRange: From: 1024 To: 65535 rPublicSubnetOutbound: Type: AWS::EC2::NetworkAclEntry Properties: NetworkAclId: !Ref rPublicSubnetNacl Egress: true RuleNumber: 10 Protocol: -1 RuleAction: 'allow' CidrBlock: '0.0.0.0/0' PortRange: From: 22 To: 65535 rPublicSubnetInbound: Type: AWS::EC2::NetworkAclEntry Properties: NetworkAclId: !Ref rPrivateSubnetNacl Egress: false RuleNumber: 10 Protocol: -1 RuleAction: 'deny' CidrBlock: '0.0.0.0/0' PortRange: From: 1024 To: 65535 rPublicSubnetOutbound: Type: AWS::EC2::NetworkAclEntry Properties: NetworkAclId: !Ref rPrivateSubnetNacl Egress: true RuleNumber: 10 Protocol: 6 RuleAction: 'allow' CidrBlock: '0.0.0.0/0' PortRange: From: 1024 To: 65535 rNaclPublicSubnetAssociation1: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPublicSubnetNacl Subnet: !Ref rPublicSubnet1 rNaclPublicSubnetAssociation2: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPublicSubnetNacl Subnet: !Ref rPublicSubnet2 rNaclPublicSubnetAssociation3: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPublicSubnetNacl Subnet: !Ref rPublicSubnet3 rNaclPrivateSubnetAssociation1: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPrivateSubnetNacl Subnet: !Ref rPublicSubnet1 rNaclPrivateSubnetAssociation2: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPrivateSubnetNacl Subnet: !Ref rPrivateSubnet2 rNaclPrivateSubnetAssociation3: Type: AWS::EC2::SubnetNetworkAclAssociation Properties: NetworkAclId: !Ref rPublicSubnetNacl Subnet: !Ref rPrivateSubnet3 Outputs: VPC: Description: "VPC Id" Value: !Ref VPC PublicSubnets: Description: "Public subnets" Value: !Join [ ",", [ !Ref rPublicSubnet1, !Ref rPublicSubnet2, !Ref rPublicSubnet3 ]] PrivateSubnets: Description: "Private subnets" Value: !Join [ ",", [ !Ref rPrivateSubnet1, !Ref rPrivateSubnet2, !Ref rPrivateSubnet3]] The above code snippet creates a VPC with public and private subnets, NACLs, Internet Gateway, NAT Gateway, and Route Tables. All CIDR ranges of subnets in a VPC should be a subset of a VPC CIDR range. In this example, the VPC creates a '/22 CIDR block' (with 1024 IPs) with three availability zones, and each availability zone has a set of public and private subnets. All our public subnets will be created using the '/26 CIDR' block, and all our private subnets will be created using the '/24 CIDR' block. Public Subnets Each Public subnet has its own route table attached to route the internet traffic through the Internet gateway. The resources deployed in EC2 Instances in public subnets can have their own public IPs or Elastic IPs (EIPs) that have a NAT attached to their Elastic Network Interface (ENI). Private Subnets AWS resources deployed in the private subnets are assigned only private IPs, and these resources cannot directly access the internet. Infrastructure in a private subnet gets access to resources or users on the Internet through a NAT Gateway. An AWS NAT gateway is a fully managed and highly available service (in a given availability zone). To provide high availability across the availability zones (AZs), it is recommended to have a minimum of two NAT gateways in different AZs. In the event that one AZ should become unavailable, the design choice of having two NAT gateways in two different AZs allows us to switch to an available NAT gateway. Applications hosted on instances living within a private subnet can have different access requirements. Some services may require access to the Internet, whereas other services may require access to databases, applications, and users that are hosted on-premises. For this type of access, AWS provides two types of services: the Virtual Gateway and the Transit Gateway. The Virtual Gateway can only support a single VPC at a time, whereas the Transit Gateway is built to simplify the interconnectivity of tens of hundreds of VPCs, and then aggregate their connectivity to resources residing on-premises. Given that, we are looking at the VPC as a single unit of the network. All diagrams below contain illustrations of the Virtual Gateway, which acts as a WAN concentrator for our VPC. AWS provides two options for establishing private connectivity between our VPC and on-premises network: AWS Direct Connect and AWS Site-to-Site VPN. AWS Site-to-Site VPN configuration leverages IPSec, with each connection providing two redundant IPSec tunnels. AWS supports both static routing and dynamic routing (through the use of BGP). Note: BGP is recommended, as it allows dynamic route advertisement, high availability through failure detection, and failover between tunnels, in addition to decreased management complexity. Conclusion In this article, we learned how to design a VPC and subnets and defined their components. We ensured our VPC has internet and private network connectivity to the resources deployed within our VPC by defining an Internet Gateway, NAT Gateway, Virtual Gateway (VGW), and Direct Connect. We learned to describe components such as Route Table and NACLs. I hope this article helps you to understand, design, and define a virtual private cloud in simple and easy steps.

This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report In today's world, it's more important than ever to have visibility into your system's performance and health. Modern applications rely on complex microservices architectures and cloud-native technologies, like Kubernetes. Observability helps us understand not just application behavior, but also infrastructure configuration changes and dependencies, as they happen in real-time. What Is Cloud-Native Observability? Cloud-native observability is the ability to understand the health and status of a system based on the external data exposed by its elements, like containers, microservices, Kubernetes, and serverless functions. Cloud-native observability is built on three postulates: monitoring, logging, and traceability. By understanding these postulates and their implications, you can begin to understand why observability is essential in modern DevOps practices. Figure 1: The three pillars of observability Monitoring, or metrics, measure the health and performance of applications and their infrastructure — these are quantifiable measurements. Metrics provide real-time alerts of resource statuses and are critical for collecting insights into how fast your application responds and/or detects early indicators of performance problems. Another pillar of observability is logging. Logging captures detailed error messages as well as stack traces. Logs are records of events, warnings, and faults that occur inside a system. They are a great source of visibility as they contain information like the time an event took place and who or what endpoint was associated with the event. By combining logs and traces, you can get the full context of a system's availability. Tracing helps to investigate an issue or performance bottleneck in a containerized or microservices-based ecosystem by collecting data of the entire journey of an application request that moves through all the layers and nodes of that system. Why Is Observability Important? Observability is very important for developers and DevOps engineers because it provides the necessary information needed to track an entire system's performance and health, troubleshoot and diagnose issues quickly, and make informed decisions about the next steps needed to fix a problem. Observability in cloud-native environments, such as Kubernetes, can help to understand what's going on within the clusters, so you can optimize compute power without compromising performance. Another great use case where observability helps is cloud cost management. To avoid unnecessary compute costs, you need to monitor clusters and understand your application's resource usage and needs. Security issues and vulnerabilities can be detected quickly with good monitoring and observability tools in place. Common Challenges in Observability and Monitoring Kubernetes is one of the most popular open-source container runtime engines because of its versatility, reliability, and ability to provide abstraction among cloud providers. However, monitoring Kubernetes can be a quite challenging task for teams that require a diverse set of monitoring tools and solutions. High Quantity of Components and Metrics Working with Kubernetes brings its own set of challenges; it has more components than traditional infrastructures, like clusters, nodes, Pods, namespaces, Services, and more. These components produce their own metrics, which can be really helpful but also overwhelming if you don't know how to understand them. The Complexity of Microservices Modern applications can include numerous microservices, and Kubernetes needs to handle their availability and scalability by keeping track of all of them. Each Service can be distributed across multiple instances, forcing containers to move across your infrastructure as needed. Additionally, microservices need to be in constant communication with each other, which is also done inside the Kubernetes cluster. Dynamic Containers Depending on the organization, the container's life can be very short (from 10 seconds to 5 minutes). This creates something known as "pod churn" — the lifecycle through which new containers and Pods are created, used, destroyed, and recreated. Kubernetes manages these elements to make sure that there are available resources at any time and that they are allocated where needed. Figure 2: Layers of Kubernetes infrastructure With every new deployment, Kubernetes decides to move, recreate, or destroy Pods as necessary. If there is a downscaling need for the application, Pods or nodes can disappear forever. This means that once a container dies, all of the information inside is gone. Advancements in Kubernetes Observability Now that Kubernetes is more popular than ever, observability tools have also become more sophisticated. Some of the key advancements in Kubernetes observability include real-time monitoring, performance analytics, and application visibility. Emergence of Observability Tools In the traditional development lifecycle, monitoring and observability haven't been exactly part of the whole process. But, now as cloud-native systems grow, and infrastructures become more complex, observability is becoming the focus for organizations. The need to maintain system stability, reliability, and performance is what enables the growth and maturity of observability and analysis tools. In the past few years, cloud-native observability tools have made huge advancements. Many different tools have emerged for metrics monitoring, collecting data, and analyzing logs and traces from individual parts of your cluster. Achieving Cluster Visibility The need to aggregate and relate all of this observability data from different sources and get a holistic view of your entire system is much bigger now. This is why different open-source tools can be easily integrated with each other, to create complete visibility for developers. Cluster visibility tools can aggregate metrics from different microservices and applications in a data center, give insights into performance during deployments, understand how active services are behaving across the data center, pinpoint application-level issues, and provide real-time alerts to administrators. To achieve this goal, most of these tools offer native integration with monitoring systems so that you can get notified if any service goes down unexpectedly or is experiencing high load. Additionally, many of these products also have sophisticated analytics capabilities that allow analysts to drill down and understand what is happening at a microservices or application level. A Unified Observability Approach In order to achieve an even deeper level of observability, every unit of data needs to be contextualized. So every metric, trace, and log captured needs to have the complete context of what is happening in the system. Additionally, developers need to be able to monitor the whole application behavior, from the start of the delivery pipeline through the deployment. Tools that integrate seamlessly into the application environment are crucial to automate this whole observability journey. This unified observability approach has an exciting promise: to provide a deeper correlation between the three pillars and help teams define and track metrics that are important for their business requirements. The state of Kubernetes observability is constantly evolving, so it's important to stay up to date on the latest trends. This includes learning about the different types of monitoring tools that are available and choosing the best one for your needs. Popular Tools for Cloud-Native Observability In the last few years, the implementation of observability has become more accessible, as there are many open-source tools available that help developers introduce different aspects of observability into their systems. Let's look at the most popular ones: POPULAR CLOUD-NATIVE OBSERVABILITY TOOLS Tool Description Advantages Disadvantages Prometheus The most adopted open-source observability tool for event monitoring and alerting Real-time metrics and data collection Grafana integration Visualization for containerized applications Doesn't have built-in long-term storage Rudimental anomaly detection Handles only metrics, not logs or traces Has challenges with horizontal scaling OpenTelemetry This project standardizes how you collect and send telemetry data to a backend platform, such as Prometheus or Jaeger Expands the fourth pillar called "profiling" to help better understand performance The Kubernetes Dashboard can be used to visually track important info, like containers and Pods running in clusters Metrics about apps and app deployments running in the Pods, as well as resource utilization Doesn't provide backend storage Doesn't have a visualization layer Doesn't provide a storage solution Jaeger A popular choice for Kubernetes tracing; its main purpose is to monitor microservice-based systems and collect data in storage back ends of your choice Makes debugging easier by using tracing to analyze root causes and monitor distributed transactions Offers limited back-end integration cAdvisor A Kubernetes built-in monitoring feature that exists by default on every node Collects data on a node level, such as resource usage and performance characteristics, and displays them on a web-based interface Doesn't offer rich functionality, as it collects only basic utilization metrics Doesn't offer any long-term storage No deep analysis or trending metrics Handles only metrics, not logs or traces Conclusion Cloud-native observability has emerged as an important part of managing Kubernetes deployments in the cloud. By providing visibility into the performance of applications running on Kubernetes, cloud-native observability has helped to improve system performance, reliability, and security. With the advancements in open-source tools, we are lowering the barriers of implementing observability for organizations. Current offerings and standards will mature and become more established by focusing on the outcomes rather than technical implementation. Additionally, the trend of moving applications to Kubernetes is likely to increase the demand for observability solutions in the future. Kubernetes helps developers ship their apps to the cloud in a containerized matter, which is now deeply connected to the cloud-native way of working. More than just a technology, Kubernetes becomes a business decision that drives the value for the company that incorporates it, because it means that you develop and deliver apps in a cloud-native manner, making you ready for innovation. Kubernetes is most powerful when properly managed and monitored. That's why it's important to think about observability from the start. Implementing the right tools and standards for cloud-native observability can save your business valuable time and resources because it will help the organization detect incidents and resolve them in a timely manner. This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report

This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report

In an earlier blog post you saw how to use cdk8s with AWS Controllers for Kubernetes (also known as ACK), thanks to the fact that you can import existing Kubernetes Custom Resource Definitions using cdk8s! This made it possible to deploy DynamoDB along with a client application, by using cdk8s and Kubernetes. But, what if you continue using AWS CDK for AWS infrastructure and harness the power cdk8s (and cdk8s-plus!) to define Kubernetes resources using regular code? Thanks to the native integration between the AWS EKS module and cdk8s, you can have the best of both worlds! The goal of this blog post is to demonstrate that with a few examples. We will start off with a simple (nginx-based) example before moving on to a full-fledged application stack (including DynamoDB etc.). Both will be using the Go programming language which is well supported in AWS CDK as well as cdk8s. All the code discussed in this blog is available in this GitHub repo Pre-requisites To follow along step-by-step, in addition to an AWS account, you will need the following CLIs - AWS CLI, cdk8s CLI and kubectl. Also, don't forget to install AWS CDK, the Go programming language (v1.16 or above) as well as Docker, if you don't have them already. Keeping It Simple With Nginx on EKS As with most things in life, there are two ways - the easy way or the hard way ;) You will see both of them! Let's try things out first, see them working and then look at the code. To start off, clone the repo and change to the right directory: Shell git clone https://github.com/abhirockzz/cdk8s-for-go-developers cd cdk8s-for-go-developers/part6-cdk-eks-cdk8s/cdk-cdk8s-nginx-eks To set up everything, all you need is a single command: Shell cdk deploy you can also use cdk synth to generate and inspect the Cloud Formation template first You will be prompted to confirm. Once you do that, the process will kick off - it will take some time since lots of AWS resources will be created, including VPC, EKS cluster etc. Feel free to check the AWS Cloud Formation console to track the progress. Once the process is complete, you need to connect to the EKS cluster using kubectl. The command required for this will be available as a result of the cdk deploy process (in the terminal) or you can refer to the Outputs section of the AWS Cloud Formation stack. Once you've configured kubectl to point to your EKS cluster, you can check the Nginx Deployment and Service. Shell kubectl get deployment # output NAME READY UP-TO-DATE AVAILABLE AGE nginx-deployment-cdk8s 1/1 1 1 1m nginx-deployment-cdk 1/1 1 1 1m You will see that two Deployments have been created - more on this soon. Similarly, if you check the Service (kubectl get svc), you should see two of them - nginx-service-cdk and nginx-service-cdk8s. To access Nginx, pick EXTERNAL-IP of any of the two Services. For example: Shell APP_URL=$(kubectl get service/nginx-service-cdk -o jsonpath="{.status.loadBalancer.ingress[0].hostname}") echo $APP_URL # to access nginx (notice we are using port 9090) curl -i http://$APP_URL:9090 If you get a Could not resolve host error while accessing the LB URL, wait for a minute or so and re-try Behind the Scenes Let's look at the code now - this will clarify why we have two Nginx Deployments. Thanks to AWS CDK, VPC creation is a one-liner with awsec2.NewVpc function and creating an EKS cluster isn't too hard either! Go func NewNginxOnEKSStack(scope constructs.Construct, id string, props *CdkStackProps) awscdk.Stack { //... vpc := awsec2.NewVpc(stack, jsii.String("demo-vpc"), nil) eksSecurityGroup := awsec2.NewSecurityGroup(stack, jsii.String("eks-demo-sg"), &awsec2.SecurityGroupProps{ Vpc: vpc, SecurityGroupName: jsii.String("eks-demo-sg"), AllowAllOutbound: jsii.Bool(true)}) eksCluster := awseks.NewCluster(stack, jsii.String("demo-eks"), &awseks.ClusterProps{ ClusterName: jsii.String("demo-eks-cluster"), Version: awseks.KubernetesVersion_V1_21(), Vpc: vpc, SecurityGroup: eksSecurityGroup, VpcSubnets: &[]*awsec2.SubnetSelection{ {Subnets: vpc.PrivateSubnets()}, DefaultCapacity: jsii.Number(2), DefaultCapacityInstance: awsec2.InstanceType_Of(awsec2.InstanceClass_BURSTABLE3, awsec2.InstanceSize_SMALL), DefaultCapacityType: awseks.DefaultCapacityType_NODEGROUP, OutputConfigCommand: jsii.Bool(true), EndpointAccess: awseks.EndpointAccess_PUBLIC()}) //... Nginx on Kubernetes, the Hard Way! Now we look at two different ways of creating Nginx, starting with the "hard" way. In this case, we use AWS CDK (not cdk8s) to define the Deployment and Service resources. Go func deployNginxUsingCDK(eksCluster awseks.Cluster) { appLabel := map[string]*string{ "app": jsii.String("nginx-eks-cdk"), } deployment := map[string]interface{}{ "apiVersion": jsii.String("apps/v1"), "kind": jsii.String("Deployment"), "metadata": map[string]*string{ "name": jsii.String("nginx-deployment-cdk"), }, "spec": map[string]interface{}{ "replicas": jsii.Number(1), "selector": map[string]map[string]*string{ "matchLabels": appLabel, }, "template": map[string]interface{}{ "metadata": map[string]map[string]*string{ "labels": appLabel, }, "spec": map[string][]map[string]interface{}{ "containers": { { "name": jsii.String("nginx"), "image": jsii.String("nginx"), "ports": []map[string]*float64{ { "containerPort": jsii.Number(80), }, }, }, }, }, }, }, } service := map[string]interface{}{ "apiVersion": jsii.String("v1"), "kind": jsii.String("Service"), "metadata": map[string]*string{ "name": jsii.String("nginx-service-cdk"), }, "spec": map[string]interface{}{ "type": jsii.String("LoadBalancer"), "ports": []map[string]*float64{ { "port": jsii.Number(9090), "targetPort": jsii.Number(80), }, }, "selector": appLabel, }, } eksCluster.AddManifest(jsii.String("app-deployment"), &service, &deployment) Finally, to create this in EKS we invoke AddManifest (think of its as the programmatic equivalent of kubectl apply). This works, but there are a few gaps in this approach: We are not able to reap the benefits of Go which is a strongly typed language. That's because the API is loosely typed, thanks to map[string]interface{} everywhere. This makes it very error-prone (I made a few mistakes too!) The verbosity is apparent as well. It seems as if we are writing YAML in Go - not too much of an improvement! Is There a Better Way..? Let's look at the second function deployNginxUsingCDK8s - by the name, it's obvious that we used cdk8s, not just CDK) Go func deployNginxUsingCDK8s(eksCluster awseks.Cluster) { app := cdk8s.NewApp(nil) eksCluster.AddCdk8sChart(jsii.String("nginx-eks-chart"), NewNginxChart(app, "nginx-cdk8s", nil), nil) } This looks "too easy" to be true! But it's made possible due to the inter-operability between CDK and cdk8s. What this implies is that, you can use define Kubernetes resources using cdk8s Charts and apply them to an EKS cluster created with CDK (this makes it sort of a hybrid system). The hero of our story is the AddCdk8sChart function, which accepts a construct. Construct (remember, everything is a construct!). In this case, the Construct happens to be a cdk8s.Chart that's returned by NewNginxChart function - so let's take a look at that. Go func NewNginxChart(scope constructs.Construct, id string, props *MyChartProps) cdk8s.Chart { //.... dep := cdk8splus22.NewDeployment(chart, jsii.String("nginx-deployment"), &cdk8splus22.DeploymentProps{ Metadata: &cdk8s.ApiObjectMetadata{ Name: jsii.String("nginx-deployment-cdk8s")}) dep.AddContainer(&cdk8splus22.ContainerProps{ Name: jsii.String("nginx-container"), Image: jsii.String("nginx"), Port: jsii.Number(80)}) dep.ExposeViaService(&cdk8splus22.DeploymentExposeViaServiceOptions{ Name: jsii.String("nginx-service-cdk8s"), ServiceType: cdk8splus22.ServiceType_LOAD_BALANCER, Ports: &[]*cdk8splus22.ServicePort{{ Port: jsii.Number(9090), TargetPort: jsii.Number(80)}}) return chart } If you have worked with cdk8s (and Go) or read some of my previous blogs on this topic, this should look familiar - a strongly-typed, compact and expressive API! I don't even need to walk you through this since it's so readable - we use cdk8s-plus to create an Nginx Deployment, add the container info and finally expose it via a Service so that we can access the Nginx from outside of EKS. This was a simple enough example to help bring out the difference between the two approaches. The next scenario is different - in addition to the EKS cluster, it has DynamoDB and a URL shortener application that will be deployed to EKS. End-to-End Example: Dynamodb Along With an Application on EKS Instead of creating a new EKS cluster from scratch, we will re-use the existing cluster created as a result of the previous example - this is a good opportunity to take a look at how you can reference an existing EKS cluster in your CDK code. As expected, we need to create DynamoDB table as well. Just like in the previous example, let's try out the solution first. Change into the right directory first: Shell cd part6-cdk-eks-cdk8s/cdk-cdk8s-dynamodb-app-eks Since the URL shortener application has to make API calls to DynamoDB, we need to configure IAM Roles for Service Accounts (also known as IRSA). Refer to the documentation for more information Define IAM Roles for the Application Start by creating a Kubernetes Service Account: YAML kubectl apply -f - <<EOF apiVersion: v1 kind: ServiceAccount metadata: name: eks-dynamodb-app-sa EOF To confirm - kubectl get serviceaccount/eks-dynamodb-app-sa -o yaml Set your AWS Account ID and OIDC Identity provider as environment variables: Shell ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text) export EKS_CLUSTER_NAME=<enter cluster name> export AWS_REGION=<enter region e.g. us-east-1> OIDC_PROVIDER=$(aws eks describe-cluster --name $EKS_CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///") Create a JSON file with Trusted Entities for the role: JSON read -r -d '' TRUST_RELATIONSHIP <<EOF { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_PROVIDER}:aud": "sts.amazonaws.com", "${OIDC_PROVIDER}:sub": "system:serviceaccount:default:eks-dynamodb-app-sa" } } } ] } EOF echo "${TRUST_RELATIONSHIP}" > trust.json Check - cat trust.json Now, create the IAM role: Shell export ROLE_NAME=dynamodb-app-irsa aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document file://trust.json --description "IRSA for DynamoDB app on EKS" We will need to create and attach a policy to the role since we only want to allow PutItem and GetItem operations from our application. Here is the policy JSON file: JSON { "Version": "2012-10-17", "Statement": [ { "Sid": "PutandGet", "Effect": "Allow", "Action": [ "dynamodb:PutItem", "dynamodb:GetItem" ], "Resource": "arn:aws:dynamodb:*:*:table/urls" } ] } Create and attach the policy to the role we just created: Shell aws iam create-policy --policy-name dynamodb-irsa-policy --policy-document file://policy.json aws iam attach-role-policy --role-name $ROLE_NAME --policy-arn=arn:aws:iam::<enter AWS account ID>:policy/dynamodb-irsa-policy Finally, we need to associate the IAM role and Service Account: Shell kubectl annotate serviceaccount -n default eks-dynamodb-app-sa eks.amazonaws.com/role-arn=arn:aws:iam::<enter AWS account ID>:role/dynamodb-app-irsa Get the EKS kubectl role ARN To reference the existing EKS cluster in AWS CDK, you need the EKS cluster name and kubectl role ARN. You can find the role ARN in the Outputs section of the AWS Cloud Formation stack. We are ready to deploy the application using CDK. Set the required environment variables followed by cdk deploy: you can also use cdk synth to generate and inspect the Cloud Formation template first Shell export EKS_CLUSTER_NAME=<enter name of EKS cluster> export KUBECTL_ROLE_ARN=<enter kubectl role ARN> export SERVICE_ACCOUNT_NAME=eks-dynamodb-app-sa export APP_PORT=8080 export AWS_REGION=<enter region e.g. us-east-1> cdk deploy CDK (and cdk8s) will do all the heavy lifting (we will look at the code very soon): A New DynamoDB table will be created The docker image for our application will be built and pushed to ECR Kubernetes resources for the URL shortener application will be deployed to the existing EKS cluster Once the stack creation is complete, check the Kubernetes Deployment and Service: Shell kubectl get deployment/dynamodb-app kubectl get pods kubectl get service/dynamodb-app-service Testing the URL shortener service is easy. But I will not repeat it here since it's already covered in [this blog](Write your Kubernetes as GO code. All you need is the load balancer URL to access the service and use your browser or curl to save and access URLs. Back To Exploring Go Code Again Within the stack, we define the DynamoDB table (using awsdynamodb.NewTable) along with the docker image for our application (with awsecrassets.NewDockerImageAsset) Go func NewDynamoDBAppStack(scope constructs.Construct, id string, props *CdkStackProps) awscdk.Stack { //... table := awsdynamodb.NewTable(stack, jsii.String("dynamodb-table"), &awsdynamodb.TableProps{ TableName: jsii.String(tableName), PartitionKey: &awsdynamodb.Attribute{ Name: jsii.String(dynamoDBPartitionKey), Type: awsdynamodb.AttributeType_STRING, }, BillingMode: awsdynamodb.BillingMode_PAY_PER_REQUEST, RemovalPolicy: awscdk.RemovalPolicy_DESTROY, }) appDockerImage := awsecrassets.NewDockerImageAsset(stack, jsii.String("app-image"), &awsecrassets.DockerImageAssetProps{ Directory: jsii.String(appDirectory)}) //... Then comes the interesting part where we get the reference to our existing EKS cluster and use AddCdk8sChart (just like before) to deploy the application to EKS. Go //... eksCluster := awseks.Cluster_FromClusterAttributes(stack, jsii.String("existing cluster"), &awseks.ClusterAttributes{ ClusterName: jsii.String(eksClusterName), KubectlRoleArn: jsii.String(kubectlRoleARN)}) app := cdk8s.NewApp(nil) appProps := NewAppChartProps(appDockerImage.ImageUri(), table.TableName()) eksCluster.AddCdk8sChart(jsii.String("dynamodbapp-chart"), NewDynamoDBAppChart(app, "dynamodb-cdk8s", &appProps), nil) The NewDynamoDBAppChart function defines the Deployment and Service. Unlike the earlier Nginx example which had static values, this application takes in dynamic values - specifically the DynamoDB table name (which is used as the container environment variable TABLE_NAME). Also, notice the fact that we explicitly add the name of the Kubernetes service account (for IRSA) that we had created in the previous step. Go func NewDynamoDBAppChart(scope constructs.Construct, id string, props *AppChartProps) cdk8s.Chart { //... dep := cdk8splus22.NewDeployment(chart, jsii.String("dynamodb-app-deployment"), &cdk8splus22.DeploymentProps{ Metadata: &cdk8s.ApiObjectMetadata{ Name: jsii.String("dynamodb-app")}, ServiceAccount: cdk8splus22.ServiceAccount_FromServiceAccountName( chart, jsii.String("aws-irsa"), jsii.String(props.serviceAccountName))}) container := dep.AddContainer(//.. omitted for brevity) container.Env().AddVariable(jsii.String("TABLE_NAME"), cdk8splus22.EnvValue_FromValue(props.tableName)) container.Env().AddVariable(jsii.String("AWS_REGION"), cdk8splus22.EnvValue_FromValue(&props.region)) dep.ExposeViaService(//.. omitted for brevity) return chart } Wrap up That's all for this blog! We started off with a simple example to showcase the integration between AWS CDK and cdk8s and how easy it makes things (compared to just using CDK to deploy apps to EKS). Then, we moved on to explore a full-fledged scenario where you deployed the infrastructure (DynamoDB etc.) along with the client application on EKS. Happy Building!

The world of software development is changing faster than ever. The need for faster release cycle times has led to a proliferation of virtualization technologies. Virtual machines (VMs) have been around for many years, but containers have recently overshadowed them because VMs are slower and not as lightweight as containers. With the popularity of microservices, containers have emerged as an excellent alternative to virtual machines for developing, testing, and deploying applications in a lightweight environment. There comes Docker, an open-source technology that uses Linux containers to virtualize apps and other software processes so they can run independently and more securely in another container as if they were separate computer programs on the same device. Docker packages software applications in virtual containers so they can be shipped, deployed, and run quickly and efficiently. These containers are like virtual lockers for your application. They package your application and its dependencies so it can be shipped and run anywhere transparently. That means you can have the same user experience from your development, testing, staging, or production systems. Docker Extensions Docker announced Docker extensions at DockerCon 2022. The idea of Docker extensions is to help developers expand their capabilities by using third-party tools, just like plugins. You can enable the extensions available from the extensions tab on the Docker desktop with a single click. The extensions improve developer productivity and smoothen the workflows. Developers can use their favorite tools from a centralized place, i.e. Docker desktop. Before exploring the various interesting extensions, go to your Docker desktop and enable extensions from the preferences. Today, we will see some must-have Docker extensions for developers to speed up their software development. DroneCI We all know that continuous integration (CI) is the first official step to doing DevOps. When developers write the code, they store it in a source control management tool like Git. Then, they test and build the code with the help of a continuous integration tool. CI is an essential part of every DevOps approach. DroneCI is an open-source CI tool that is very easy to set up and use. Now that we have this as an extension, you will be able to do CI from your laptop. You just need a yml file to define your steps, and you are done. Let us have a simple Node.js application to work with our plugins. Clone the sample application. kind: pipeline type: docker name: default platform: os: linux arch: arm64 steps: - name: message image: busybox commands: - echo “Hello Captain Canary”! - name: test image: node commands: - npm install - npm test You can directly install the Drone CI extension from the Docker desktop extensions tab. Once installed, click on the Drone CI extension and you will be presented with the below dashboard: As we have not configured any pipelines yet, you should see a blank dashboard with no pipelines imported. Click on 'Import Pipelines' to import your project. The extension will show you all the projects that have the .drone.yml in them. So, import a project to try from your local. To show you an example, I imported my Simple-Node-App project, which is a simple node.js application that includes simple tests. The next step is to run the pipeline, and for this, we need to add some simple settings that we will be doing once we click on the run/play button; you see the below actions: Add the required fields such as steps to run, a secret file if you have one (which you can specify in a secret.txt file), an environment file, etc. Once you run your application, you will see the pipeline getting executed. vcluster Working with Kubernetes can be highly overwhelming. As it involves a vast learning curve, developers spend more time and effort understanding this platform. It is all worth it, but what if there are tools that aid developers and help them increase productivity? vclusters is one such tool with which you can create virtual clusters on top of any Kubernetes cluster. Instead of creating full-blown clusters that might just shoot up your cloud bill, the vclusters are cheap, with the added advantages of multi-tenancy. This will benefit anyone required to set up different development and testing configurations. Make sure to enable Kubernetes from the Docker desktop preferences. The vcluster extension is readily available on the Docker desktop extensions tab and can be installed with a click. Once the extension is installed, you will see a simple dashboard to create the virtual cluster. You can then click on 'Create new cluster.' Name your cluster, fill in the required fields, and click on 'Create.' Once you fill in all the required fields, you should be able to see your cluster getting created. After some time, you should see it running. You can verify the running cluster using vcluster CLI. The command to list your vclusters is: vcluster list Microcks Microcks is an open-source platform to mock and test APIs and microservices. You can deploy mocks and experiment with your rest APIs locally. Now that the Docker extension is available, you can efficiently mock and test event-based APIs. Microcks extension is readily available on the Docker desktop and can be easily installed with a click. Click on the ‘Launch Microcks’ tab: The Microcks starts running, and you can access the localhost:8080 to test your APIs. Let’s go to localhost:8080, and you should see the Microcks dashboard. Click on the 'APIs' and 'Services' tabs: Choose 'REST API' and click 'Next.' Name the API, Version and Resource and click 'Next.' Add the dummy JSON Reference Payload as shown below for this experiment and click 'Next.' The last step is to Review, and once the data added by you is correct, click 'Next.' You should see your just-created API listed. Click on it to see the CRUD operations listed with endpoints. Let’s test with Get. Copy the link and open the endpoint on a local browser and you should see something like this: Conclusion Docker is a must-know platform for anybody working with software development. Over the years, Docker has built a vast community and support. This is an excellent initiative by Docker to help developers to streamline their work around software development and deployment. The exciting fact is you can build your own custom-made extension. We have several extensions on and off the extensions marketplace. From continuous integration to delivery, we have all the extensions we need. The above-listed extensions can significantly impact your software delivery pipeline when used wisely. Please try them and increase your speed and productivity.

Log data, as it grows in importance and its sheer volume – is increasingly looking like analytical data. Small, semi-structured datasets in high volumes, critical insights to be captured from the data – the boundary between a log and an analytical event is as thin as ever. Organisations are adapting by switching to OLAP stores like ClickHouse for their log storage. But OLAP platforms are essentially databases, built for BI-type use cases. This leaves a huge gap in the overall experience. Most important logging features like alerts, correlation, deep diving into an incident, and much more are not at all available. This gap led to Parseable, a modern, scalable, cloud-native log storage system that works and scales like an OLAP system, but behaves like a log storage and observability platform. Logging is a routine activity for any software-enabled business. Logs help keep the business running and customers happy. With the patterns I discussed above, there is a clear need for a developer-friendly, easy-to-deploy/maintain platform that doesn't get in the way of developers doing their job, but rather paves the way. Parseable is a simple, efficient log storage and observability platform. We're aiming at: Ownership of data and content Simplicity in deployment and usage, deploy anywhere. Kubernetes-native. OLAP DB-like architecture, logging platform-like behaviour. Head over to Parseable community Slack with any questions, comments, or feedback! Getting Started on Kubernetes With the introductions out of the way, let's take a look at how to set up and run Parseable. We'll see how easy it is to get to searchable logs with Parseable. Prerequisites S3 or compatible buckets as the storage layer. Kubectl and Helm installed on your machine. Install on Kubernetes with Helm We'll use a Parseable Helm chart to deploy the Parseable image. To get started in demo mode, use the commands: Shell helm repo add parseable https://charts.parseable.io/ kubectl create namespace parseable helm install parseable parseable/parseable --namespace parseable --set parseable.demo=true Note that we're using Parseable in demo mode here (--set parseable.demo=true), which means Parseable will use a publicly hosted bucket. We use this to get an idea of the Parseable working, but make sure to configure Parseable to use your own S3 bucket. Refer to the configuration options section for details on how to set up your own bucket. Access the GUI Let's first expose the Parseable service (created by the Helm chart) to localhost, so it is accessible on localhost:8000. Shell kubectl port-forward svc/parseable -n parseable 8000:80 You should now be able to point your browser to http://localhost:8000 and see the Parseable login page. In demo mode, the login credentials are set to parseable and parseable. Log in with these credentials. Right now, you may see an empty page, because we've not sent any data. Let's send some data to the demo instance now. Sending and Querying Logs Before sending logs, we need to create a log stream. Use curl to create a log stream like shown below, but make sure to replace <stream-name> with a name you intend to use for the log stream. Shell curl --location --request PUT 'http://localhost:8000/api/v1/logstream/<stream-name>' \ --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ==' Next, let's send log data to this log stream. Shell curl --location --request POST 'http://localhost:8000/api/v1/logstream/<stream-name>' \ --header 'X-P-META-meta1: value1' \ --header 'X-P-TAG-tag1: value1' \ --header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ==' \ --header 'Content-Type: application/json' \ --data-raw '[ { "id": "434a5f5e-2f5f-11ed-a261-0242ac120002", "datetime": "24/Jun/2022:14:12:15 +0000", "host": "153.10.110.81", "user-identifier": "Mozilla/5.0 Gecko/20100101 Firefox/64.0", "method": "PUT", "status": 500, "referrer": "http://www.google.com/" } ]' Now that we've sent data to the Parseable demo instance, switch to the Parseable dashboard on your browser and select the log stream you just created. You should see the log data we just sent. You can also use the Query API to run SQL queries on the log data directly. Refer to the docs here. Configuration Options To configure Parseable so it uses your S3 / compatible bucket, you can set variables via the Helm command line (e.g. --set parseable.env.P_S3_URL=https://s3.us-east-1.amazonaws.com) or change the values.yaml file with proper values. This is a list of relevant variables to configure the correct S3 endpoint. Shell P_S3_URL=https://s3.us-east-1.amazonaws.com P_S3_ACCESS_KEY=<access-key> P_S3_SECRET_KEY=<secret-key> P_S3_REGION=<region> P_S3_BUCKET=<bucket-name> P_USERNAME=<username-for-parseable-login> P_PASSWORD=<password-for-parseable-login> For further details, please refer to the Parseable documentation. I hope this article helped you gain a new perspective on the log storage space. If you find Parseable interesting, please consider adding a star to the repo here: https://github.com/parseablehq/parseable

This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report If you are a developer or an engineer working in the software industry, then you have undoubtedly noticed the complexity of managing resources across different services and applications. This is especially true when your company has scaled beyond a single data center or environment. With so many stakeholders involved in various projects, it becomes difficult for everyone to keep track of where everything stands at any given time. This is where Kubernetes comes into the picture. There are several reasons why Kubernetes is gaining popularity among modern businesses: Kubernetes can improve productivity and scalability, reduce the risk of downtime, and enable continuous delivery of applications. Kubernetes comes with a rich set of features out of the box, making it easy for you to set up and manage a containerized application lifecycle from development to production. Kubernetes is open source, and its development has been driven by the community. It is also highly extensible, allowing third-party vendors to build tools and services that can integrate with Kubernetes and make it even more powerful. Kubernetes supports all major operating systems and is also compatible with most standard programming languages. Kubernetes is a powerful tool for accelerating your journey to the cloud-native world, making it easier for you to manage containerized microservices and accelerate your adoption of DevOps practices. In this article, we will see why Kubernetes is a must-adopt cloud-native tool for businesses and how it helps them speed up their software deployment. Kubernetes: King of the Cloud-Native Ecosystem Today, enterprises of all sizes are adopting cloud-native computing to accelerate their digital transformation. Kubernetes continues to be the leading provider in the cloud-native ecosystem and is becoming an essential tool for anyone looking to adopt a cloud-native strategy. Figure 1: Kubernetes architecture Image source: Nived Velayudhan, CC BY-SA 4.0 Kubernetes can help you achieve a cloud-native ecosystem by automating the process of setting up an environment where you can host your containerized applications. This can be done with just a few simple commands. Kubernetes can also help you deal with the complexity involved in managing your application resources across different services and applications that are distributed across multiple environments. Kubernetes handles the entire lifecycle of your application containers, including the orchestration, configuration, and scheduling of containers across a set of machines. In addition, Kubernetes is designed to be highly scalable and fault-tolerant, making it ideal for continuous delivery of your code and enabling you to achieve continuous delivery with less manual effort and better collaboration between your teams. The most significant benefit of Kubernetes for DevOps and the cloud-native ecosystem is that it alleviates burdens. It allows engineers to share their dependencies with IT operations. It also resolves conflicts among different environments. It allows the engineers to handle customer needs while relying on the cloud for many functioning applications. Kubernetes simplifies container tasks, such as canary deployment, rolling updates, and horizontal, vertical, and dynamic auto-scaling. It is critical to the DevOps managed services to streamline development, testing, and deployment pipelines. Kubernetes can assist with continuous application delivery by rolling out updates in a consistent and streamlined fashion. The application can be orchestrated using containers. In order to deploy multiple applications using Kubernetes, each one of them is placed in a distinct container. Since the updates can be performed in one part of the application by restarting the container without impacting the remainder of the software, they can be accomplished in one part of the app. Life Before Kubernetes It all started with big monolithic applications, which involved the complexities of managing and delivering software. It used to take months for developers to push their changes to production. There used to be severe downtimes, and businesses used to incur losses due to insufficient uptimes. It was a headache for organizations to make sure their services handled ongoing changes smoothly without affecting any services. That is when microservices came along as a boon, where a humongous monolithic application breaks down as several services, and each service is handled by a group of developers. Each microservice behaves as an individual service without affecting the other service. This helped developers as well as organizations to deploy software in chunks with increased frequency. During this time, Docker was introduced to deploy these microservices with the containerization concept. Figure 2 below shows a simple representation of microservices architecture with various microservices involved in an e-commerce website: Figure 2: Representation of microservices Docker completely revolutionized software packaging and delivery with containers, helping organizations to deliver software with speed. While containerization made a big move, there was a need for an orchestrator to manage these containers, and that is how Kubernetes was born. Consider a scenario where a start-up is making use of Docker containers. As the startup grows, the number of applications also increases and, similarly, so do the containers. It is easy to manage these containers when their number is lesser, but as the number starts to grow, it becomes difficult and you need an orchestrator to manage these containers. This is exactly where Kubernetes shines and helps you manage the containers with its unique features of scalability, security, self-healing, high availability, and much more. Figure 3: Application delivery flow Life After Kubernetes Google introduced Kubernetes in mid-2014, and later Microsoft, Red Hat, IBM, and Docker joined the community. Kubernetes made every developer's and SRE’s life easier by helping them coordinate well and manage the workloads efficiently. When used efficiently, Kubernetes: Removes downtimes Helps in application scalability according to the traffic pattern Helps in securing the application and the Secrets by incorporating security policies Allows running a plethora of applications in production Allows applications to roll back if something goes wrong Increases developer productivity by automating the tasks associated with containers. With Kubernetes, you can deploy new projects more quickly. A reusable pipeline and new loads as code modules can now be distributed across a variety of projects. It can be difficult for development teams to keep tabs and monitor infrastructure operations if they are not automated. Unexpected traffic spikes or power outages, for example, might occur. When the application goes down, it is not a good sign for the organization. With Kubernetes, you can automate scaling and update patches to address such issues. Let’s consider an example to help understand the importance of Kubernetes in the cloud-native ecosystem. Let’s say you have a production app that handles customer orders. You don’t want to push new code to that app unless it’s been thoroughly tested and meets certain criteria. Traditionally, you would wait until your testing team has confirmed that the new code can be deployed safely, and then you would wait for an operations engineer to perform the deployment manually. While this model works, it’s incredibly time-consuming. This is especially true if you have multiple apps that require this type of attention. If your app fails, you might need to wait several days before retrying the deployment. This causes your organization to lose out on valuable production time while you wait for the system to be ready again. With Kubernetes, you can use continuous delivery to deploy new code to your production app every time someone pushes a new version to the code repository. This ensures that nothing is deployed to production unless it meets your organization’s deployment criteria. Kubernetes also has a built-in self-healing mechanism that can quickly detect when nodes go offline and bring them back online. Kubernetes Usage: Notable Case Studies Now, Kubernetes is being used by all the big organizations that like to deploy their software fast with scalability. Kubernetes has become a de facto container orchestration tool in the cloud-native world. Let's look at some notable examples of Kubernetes usage in different organizations: The German automaker Mercedes-Benz makes use of 900 Kubernetes clusters to support its wide range of products across the globe. PayPal recently started scaling Kubernetes to over 4,000 nodes and 200,000 Pods. HBO found that in times of peak demand for Game of Thrones, it was running out of available IP addresses to help deliver the content to viewers, and HBO chose Kubernetes to solve this problem along with scalability issues. "We went from not running a single service inside of a container to hosting all of the Games of Thrones Season Seven with Kubernetes," Illya Chekrygin, Senior Staff Engineer at HBO, told the 2017 KubeCon audience.. Kubernetes helped Tinder Engineering drive innovations toward containerization and low-touch operation through immutable deployment. Application build, deployment, and infrastructure would be defined as code. They were also looking to address the challenges of scale and stability, and they solved interesting challenges by migrating 200 services and running a Kubernetes cluster at scale. Pinterest used Kubernetes to scale its workloads, and by the end of 2020, they orchestrated 35,000+ Pods with 2,500+ nodes in their Kubernetes clusters. Conclusion From its origin as an open-source project within Google to becoming a leading force in the enterprise cloud-native space, Kubernetes has come a long way. Today, enterprises and developers from all over are finding new ways to leverage Kubernetes for operations and application development. Kubernetes has become the standard approach for deploying and managing containerized applications. The Google-backed project has grown significantly in adoption since its inception. If you’re looking to accelerate your organization’s digital transformation journey, you should consider using Kubernetes. While it’s not a tool that every organization will need, it is an essential piece of any modern IT toolkit. Kubernetes has gained a special place in the cloud-native ecosystem, and in fact, it has become a synonym for the word "cloud-native." This is an article from DZone's 2022 Kubernetes in the Enterprise Trend Report.For more: Read the Report

This article is a part of a series that looks at serverless from diverse points of view and provides pragmatic guides that help in adopting serverless architecture, tackling practical challenges in serverless, and discussing how serverless enables reactive event-driven architectures. The articles stay clear of a Cloud provider serverless services, only referring to those in examples (AWS being a common reference). The articles in this series are published periodically and can be searched with the tag “openupserverless.” In this article, we discuss considerations of running serverless at scale. The article also talks about the architectures currently popular for achieving this Serverless Architecture for Scale use cases and how and when we can optimally use the Serverless architecture for scale with AWS Lambda and Dynamo DB exemplar scalable options. Why Talk About Serverless for Scalability? When we talk about Scalability, we know that Scalability, Availability, and Performance can be best optimized in many cases with a Client fully controlled On-premises infrastructure setup. But when there are tighter budget approvals, the round-the-clock maintenance and billing costs of the operating systems management, server deployments, admins, etc., become problematic and are perceived as very high costs. Pay as you Go, else DONT and save manifolds! In serverless architecture, you need to optimize the code to trigger functions in response to events, and you’re charged only for the time when your trigger event is active. A sudden spike in load will lead to more events and, of course, more cost, but only for when the spike has occurred. Hence it significantly reduces operational cost and complexity associated with the same, and the developers can concentrate on only providing productive results. In the case of serverless architecture, the cloud service provider handles 100% of server management, which includes traffic optimization, development environment updates, resource allocation, and backend configuration. So it is like you are getting zero to minimal overhead at reduced costs! Here, during periods of peak load, the serverless function can easily scale in response to the multiple concurrent requests at peak. Design Model Considerations for Your Serverless Applications at Scale Design 1: Serverless Synchronous Model In the Serverless Synchronous model, the serverless function will scale corresponding to the number of requests/events resulting in the corresponding number of outputs. However, in this case, if the target output system or the downstream system is not designed or modeled to handle the extra load, it may crash. It is suitable in case of known target outputs at peak load. If we know our systems are capable of handling or storing the peak load at scale, then this model is probably very simplistic, fast, least expensive, and productive in its approach. Design 2: Asynchronous Model More Suited for Cloud Native Design In this model, we consider decoupling our architecture and using an intermediary resilient service to store the incoming bombardment of requests without any data loss. For the intermediary storage service, we can consider any Event supporting models like queues, e.g., IBM MQ, Amazon SQS, or even durable storage like Amazon S3/Dynamo DB, etc. or any Streaming service like Apache Kafka, Amazon Kinesis, etc. In this design, we can poll for the requests from the Queue/data store, for example, and then have a batch defined where the serverless functions can work on a defined batch size of requests per iteration and optimally send the results to the target systems. In this model, we have much better control to handle the target systems' limitations, if any, while ensuring the reliability of the messages. The Design Models Are Explained With AWS Lambda, Dynamo DB, and SES Serverless Options as Examples Consider a large Airline service provider who may have to deal with millions of seasonal flight bookings varying as per different holiday seasons. If the airline provider considers Design 1 elaborated above, a possible solution may look like the one below, with AWS Gateway talking to multiple backends like AWS Lambda, Dynamo DB, SES, and S3. Design 1 Example: Here, we have used AWS API Gateway to route the messages to concurrent Lambda functions, which then output the results to Amazon DynamoDB, a NoSQL database that is used for storing data here and can scale as per the scaled loads, Amazon SES, a cost-effective, flexible, and scalable email service that enables the airline service provider to send email notifications to the users as needed and Amazon S3 is used for hosting the static website content like HTML, media files, CSS, JavaScript which acts as a front end in the airline client’s browser. Here, the three target systems – Dynamo DB, SES, and S3 are well-scalable products by themselves with their own scaling capacities; hence the design would be a good fit to be considered with CloudWatch alarms set-up to trigger another set of Lambda functions to send notification messages to the users as needed in case of any alerts from the alarms. We can leverage CloudFront to distribute the static content securely and globally and provide a better performance, as shown above. Caching static content with CloudFront gives you the performance and scale you need to provide viewers with a fast and reliable experience. CloudFront scales automatically as globally-distributed clients get the requested data available right at the edge where your users are located. Amazon CloudFront can be used for dynamic content and to leverage features like caching. Another Scenario could be that of a Retail chain that expects peak loads only in case of a Super-Sales period but, of course, cannot afford to be dependent on the target system capacity as this Sales day could be that of Black Friday where the expected throughput may be manifolds the normal business day. Here, we can go for an option as specified below by introducing Amazon SQS, for example, to store the messages reliably and then trigger the Lambda function in a batch size that is well-suited for its underlying storage in the Dynamo DB scalable datastore. Design 2 Example: Lambda Concurrency vs Throughput We can do a high-level concurrency estimation of lambda based on the throughput and the function execution time as follows: Concurrency Estimation Avg Function execution time in seconds * average requests/sec = Concurrency estimate Eg. if Average execution time of request is 0.2 seconds and throughput is 5 requests/sec, then concurrency estimate = 0.2 *5 = 1 Scaling Limits in Serverless When all current containers for the required function are already in the middle of processing events, the Serverless function needs to scale out. And it does so magically without any effort! For example, the AWS Lambda function will scale out automatically to support the extra load without any need to provide for the same in advance from your side. And more so, the lambda cost for processing 500 events in parallel by 500 containers will be the same as processing one event at a time serially by one container. So are we saying we can scale out indefinitely with the serverless architecture? Obviously No. The Cloud Service Provider will be bankrupt in that case in no time! So by default, each CSP will set these limits for the number of concurrent executions across all its serverless functions per account per region. E.g. per the AWS account, this limit is currently 1000 for the Lambda functions, but we can request to get this increased by contacting the CSP Support team. If we reach this Scaling limit set by the CSP, we see Throttling errors in the execution. It is important to note since this limit is account-wide, one serverless function that has scaled out BIG can potentially impact the performance of every other function in the same account. Hence it is advisable to segregate different potential services expected to scale in different accounts to avoid these throttling issues. Concurrency Controls Another option to avoid Throttling errors due to the scaling limits described above is to configure Concurrency Controls per function at the CSP level. So in the case of AWS Lambda, we can configure it as below: Reserved Concurrency We can set Reserved Concurrency per function so that other functions cannot use its limit and cause throttling issues to this function when trying to scale out. This can usually be configured free of charge. Provisioned Concurrency With Provisioned Concurrency, we can request for some pre-provisioned environments by the CSP to immediately run and respond to expected peak loads. With Provisioned Concurrency, the CSP will take care of the provision of your environments for you without any latency of cold starts. This, however, is a chargeable option. Other Stress Points Besides throttling errors, we may see other stress points as below during this execution at scale: DB load can impact query performance and Lambda execution time. Higher the lambda execution time, the more the cost, and it can also cause API Gateway timeouts (~29 sec). Higher lambda execution time can also cause Lambda function timeouts (~15 min). Lambda burst throttling can cause API Gateway errors. Optimizations If we come across the stress points listed above, we can work on below possible optimization solutions: Lambda function -> AWS Step Functions Workflow 2. Richer workflows as below: 3. API Gateway Throttling options. 4. Convert API from Synch to Asynch and use patterns such as Polling, Webhooks, Websockets, etc. 5. Consider changing database types like Amazon Aurora Serverless data API or the use of Dynamo DB. 6. Pool and share database connections. Serverless Scaling Model The serverless function, e.g., Lambda by default, follows a horizontal scaling model by instantiating new containers to handle the extra load. We can, however, configure the serverless function to scale vertically too. E.g., Lambda functions in AWS can be manually configured to scale from 128 MB to 10 GB of RAM (currently), and the performance scales vertically proportionately.

This article is a part of a series that looks at serverless from diverse points of view and provides pragmatic guides that help in adopting serverless architecture, tackling practical challenges in serverless, and discussing how serverless enables reactive event-driven architectures. The articles stay clear of a Cloud provider serverless services, only referring to those in examples (AWS being a common reference). The articles in this series are published periodically and can be searched with the tag “openupserverless”. Overview The Serverless compute model has reached the “Early Adopter” stage in the Hype Curve and moving very fast towards attending the “Early Majority” stage. While the evolution of Serverless has been rapid and phenomenal, enterprises are lacking in strategy in adopting Serverless and leveraging its advancement in technology and architecture for an efficient IT ecosystem. This article attempts to provide simplified decision guidance for Serverless Architecture Adoption while not prescribing specific detailed guidance on the decision matrix of specific FaaS, BaaS, and other serverless services provided by Cloud Service Providers (CSPs) e.g., Serverless Databases, API Gateways or Edge services. Characteristics of Serverless Candidates Before we delve into the detailed Serverless adoption guidance, it is important to understand the characteristics of Serverless Candidates that assist in the evaluation process for adoption. The table below provides some technology-neutral traits of application or workload patterns that are an easy fit into serverless. These traits are the building blocks of more complex serverless patterns, solutions, and architectures These characteristics work with a combination and are not exclusive. S. No. Characteristic Detail Rationale 1 Short Process Running Time Functions, applications or services that run for a finite and short time, typically CPU or minimal IO bound. Functions, applications or services' running time affect two important factors: Cost: Cost is based on execution time (in ms).Concurrency: Processing time affects the throughput and concurrency. The process with a Long running time reduces the throughput. synchronous flows may fail due to time-outs. The following example explains the relation between concurrency, a process running time, and throughput.Concurrency = 2, processing time is 400 ms, and throughput will be 5. If there are more than 5 requests, these will be queued up or throttled. Simply put, Concurrency = Process Running Time x Request rate per second The process running time should be short consistently to keep the cost and concurrency in check. 2 Quick Start up Lightweight Applications, functions, or services with simple dependencies that use asynchronous execution with quick start-up time. Start-up time may include: Retrieve Source code Start the deployment form factor e.g., container Bootstrap any initialization as required After successful bootstrap and initialization, applications, functions, or services can accept requests to execute Typically; Start-up time influences two of serverless challenges 1. Cold Starts 2. Traffic Patterns. Long Start-up time affects the ability to provide consistent throughput. For a traffic pattern that matches a period of idle time (no requests) followed by a sudden spike or burst. During the spike of burst time, functions with significant cold start fail to serve the requests. Inconsistent traffic scenarios, a specified number of function instances are kept in the initialized or warm state by the platform anticipating some requests to be served. Start-up should not impact the required throughput or concurrency in any case, including different traffic scenarios. 3 Consistent and Predictable running time Applications, functions, or services that run for consistent and predictable time irrespective of the way and source of the trigger. Variation or semantics of the source of trigger or requests should not impact the running time consistency or predictability. Running time impacts the cost, concurrency (and hence throughput) as described in Short Process Running Time characteristic. 4 Consistent and Predictable Usage of Memory Serverless platform requires the memory allocation choice as input. Services that use consistent and predictable memory, irrespective of event or a request payload that triggered these, are more suitable for serverless compute form factor than the workloads for which memory requirements vary dynamically. Typically, the serverless platform takes the 'memory to be allocated' input and allocated Compute (CPU) based on it. Hence it is important to select the right memory allocation carefully as it controls the allocation of resources like compute by platform and is considered as the key lever for serverless resource allocation. 5 Loose Coupling or Decoupled patterns such as Asynchronous, Reactive, Event Driven The genesis of serverless advocates event-driven pattern that essentially provides loose coupling or decoupling. Leveraging the Reactive framework results in IO unlocking or unblocking and optimized use of computing. The paradigm of Synchronous execution has two issues i.e., thread blocking and request/execution queueing In hybrid cloud and distributed environments, this results in poor scalability. The scalability is enhanced significantly with an event-driven pattern that unlocks or unblocks the execution instances, and execution queues are managed by the event-triggering sources. As a result, scaling becomes more dynamic and efficacious. 6 Unpredictable spikes and bursts traffic patterns Traffic patterns that cannot be predicted inherently comes with sporadic spikes or bursts. The premise of cloud-native implementation addresses this through elastic scalability making the services resilient and responsive in such situations. Serverless compute services are automatically scaled in response to request demand by the platform. Serverless brings this unique differentiation. Without any auto-scaling rules and complexity, the serverless platform handles the request spike and burst, given the specified limit of concurrency and burst capacity. The platform can provide the required number of instances for serverless functions and services to address request spikes if the cold start issue is well managed. 7 Stateless principle of cloud native workloads One of the key tenets of 12 factor Cloud Native application is Statelessness. Horizontal scaling helps improve performance by making the required number of instances available to serve the request traffic. Dependency for maintaining state or request affinity to a specific instance leads to ineffective scaling. Due to the inherent characteristic of event-driven execution, serverless computing doesn't keep any previous context of request execution. Every instance will execute the request as if it is serving a new request. Other solutions like distributed cache or environment variables can be considered if any state or request context needs to be managed or shared. 8 Execution in parallel Parallel execution threads process a service independently without any shared or global state or context. Processes or Workloads that can be decomposed into multiple parallel independent functions are suitable for Serverless applications (e.g., processing a large set of structured and unstructured data, Big Data). These types of workloads allow execution of multiple processes as parallel functions. 9 Long Running Process that can be broken into small, short running processes Often long-running process that doesn’t maintain a resilient process state can be decomposed into a short-running process that can be executed in parallel. Moving to serverless architecture with parallelism and a scalable model for short-running processes will help in reducing sustained execution environment need resulting in lower TCO and improving agility and resiliency. 10 Minimal or no periods of inactivity Applications, services, or workload that have the minimum number of requests served without idle time. It is sometimes obvious to think that workloads that have a large number of inactivity periods and unpredictable spikes/bursts are suitable for serverless as the platform does not charge for idle time for serverless computing. However, performance is significantly impacted in such cases as the initialization, and bootstrapping context of instances are gone with the instances and cannot be reused as the platform removes these. Such cases are better suited for containers. A minimum consistent number of requests helps maintain a set of initialized and warm functions to serve any traffic. 11 Databases with unpredictable and erratic usage pattern Databases that are implemented as serverless services can scale elastically Capabilities: A) Databases as serverless services provided by platforms offer capacity-based costing where consumers are charged for the capacity consumed. B) Serverless Database services are elastic in a way that platform manages the underlying storage.For high transaction throughput requirements, the performance provided by Serverless Database services is linear. 12 Quick validation of new ideas Validating a new idea/hypothesis that requires quick PoC/PoV. Adoption of serverless removes the need for provisioning and configuring any infrastructure. So the function and services can be built rapidly and tested. 13 Continuous data streams that are not bound Data streams that are continuous and required to be processed by stream functions such as filters, maps aggregation etc. Backend Services, along with FaaS providing processing hooks, can process continuous data streams that are unbound. 14 Processing of IIOT and general IoT Sensor Data This also qualifies as Continuous data streams that are not bound. Stream processing, IoT analytic & IoT protocol support e.g., MQTT, are provided by Cloud Service Providers. 15 Simplified Operation Server maintenance is being viewed as an overhead for the organizations. The serverless platform provided managed operation services that can reduce the burden of managing the server environment and simplify operation. Candidate Architectures for Serverless There as certain architectures as follows that are more suitable for serverless adoption across Applications, Data, Integration, AI, IoT, etc. Application Reactive Systems Domain Driven Design based Microservices Strangler Transformation Data Big Data Databases that include SQL and No SQL DBs e.g., Document DB, Columnar DB, Key Value, RDBMS, Object Store Data Processing Stream processing CDC Batch ETL Integration REST API Event Driven Notifications Messaging Event Streams Workflows Processing HTTP/HTTP(s) BPM Workflows Transcribe Transcoding AI/ML IoT Event Processing Blockchain processing Cross Cutting (Security & Compliances) IAM, identity Federation Key, Certificate Management, RBAC, Secret Vaults, HSM Firewalls, DDoS, Regulatory Data Compliance IoT Device Security Cross Cutting (DevOps) CI/CD Observability Health Dashboard, Cost Management, Account Management IaC Example Use cases for Serverless Functions Some of the example use cases for serverless functions are listed below, and the list is not finite but evolving. Act on events triggered by internal and external services/sources. Schedule tasks according to a specific timetable (periodic), such as taking backups and log analysis. Implement API Management for existing services or applications. Execute app logic in response to a database change. Invoke auto-scalable mobile/API backend services. Image processing coupled with Cognitive services for visual recognition. Streaming, Image, and Video Manipulation based on targets. Perform edge analytics in response to sensor input (IoT). Extend and enhance workflows and their data with new functional logic (e.g., send notifications, tag data, add weather data). Act as a glue between different services to create a powerful pipeline. Implementation of microservices, as well as parallel computing or data processing. App requires event-based/asynchronous-based communication to implement use cases. Polling use cases, pub-sub kind of realization. Cases Unfit for Serverless Also, there are cases where Serverless may not be a suitable option to choose as follows: Workloads that need high-performance computing (HPC) with components executed in proximity. Processes that have long execution time and require Master/Worker node type of clusters for processing. Workloads that require control over the underlying infrastructure components like physical sockets or cores e.g., workload requires licenses that are bound to per-core, per-socket or per-VM. Organizations that operate in regulated industries and need compliance with running workloads in dedicated infrastructure instead of multitenant environments. Workloads that require complex and/or fine-grained auto-scaling rules with predictive or ML algorithms. Tasks are long-running or anticipate long latency (perhaps due to external connections). Functions that are complex (non-separable) or take a long time to initialize. Requirement mandates stateful sessions to implement the use cases. Functions that involve transaction management with DB and, at the same time, have requirements around rapid scaling. DB might become a bottleneck for scaling. Compliance requirements mandated by the client (For e.g., if the compliance requires scanning the underlying infrastructure, it is not possible as there are no specific target infrastructures in Serverless). Requirements on runtime version for implementation are specific (The reason being we have no control over the serverless runtimes and updates are driven by the vendor). Application architecture for serverless is vendor dependent (Potential for vendor lock-in, particularly involving platform capabilities, such as Authentication, Scaling, Monitoring, and Configuration Management). Multi-tenancy is not a preferred option when data processed are sensitive in nature. Simplified Serverless Adoption Decision Guidance Framework Based on the characteristics, architecture type, and use cases, a simple Serverless Adoption Decision Guidance framework is illustrated below. There exist various service types by CSPs for serverless implementation, primarily FaaS/BaaS and Serverless Container Platform. Key Characteristics of Serverless Platform Some of the key characteristics of Serverless Platform are listed below. Simplified Programming Model because the entire application can be described as event triggers for FaaS and BaaS and that the overall "application" can be composed from smaller serverless building blocks. Focus on front-end application Logic using short-running, single-purpose, RESTful functions: Simple (JSON) Input / Output Localized configuration via environment variables Polyglot - Choice of the programming language that suits your needs; Combine functions written in different languages. Event Driven - Multiple modes of invocation (Automated via Triggers/Messages, Manual from API invocations) Simplified Data and Service Integrations – “out-of-box” integrations with Storage (Database, Object Storage, etc.) Messaging, API Management, and other Provider Services Towards “NoOps” - Serverless platform manages operational aspects such as provisioning, deployment, auto-scaling configuration, availability, etc. Platform-provided Operational Support Services - “Built-in” support for logging and monitoring, Identity and Access Management, etc. Pay for only the Compute you use - Pricing is based on functional execution time or # of requests. Simple Decision Guide for FaaS/BaaS vs Serverless Platform It is critical to understand choose between FaaS/BaaS services from CSP or use serverless platform that can run containers. A simple decision guidance is provided below. Conclusion While Serverless Computing is evolving rapidly bring new services and capabilities beyond the typical set of use cases that adopt it currently, may organizations have significant challenges in Serverless adoption strategy fundamentally. This paper attempts to provide simplified guidance that may assists to expedite Serverless adoption.

Kubernetes networking principles are really confusing, especially if you’re a beginner. Regardless of how well I followed the official documentation, I honestly didn’t understand a single damn thing! After watching some tutorials on YouTube, I was able to write and duplicate the tutorials, but a core understanding of Kubernetes networking concepts was still missing. This blog post is my take on explaining Kubernetes networking concepts in depth and answering the following questions, which always confused me: How does a Kubernetes networking resource solve the Service Discovery problem in Kubernetes? Does a LoadBalancer service really provision a load balancer automatically? What will happen if I create this service locally? How does a production-ready Kubernetes cluster expose its applications? What is the difference between an Ingress and an Ingress controller? and more… Let’s Get Started Throughout the entire blog post, we will assume the following. You have familiarity with the structure of Kubernetes YAML resource definition. You have deployed an imaginary Kubernetes cluster on the cloud across three VMs. The VMs/nodes have the following Public IP addresses Node A (192.168.0.1) Node B (192.168.0.2) Node C (192.168.0.3) A microservices application having four services is deployed on that K8s cluster. The application is deployed in such a way that each VM has at least one replica of that service running Products Reviews Details Ratings Notations $Reviews_A$ denotes that the Reviews service is running on node A K8s denotes Kubernetes LBs denotes Load Balancers Kubernetes Cluster Let’s start by answering the why. Why Do We Need Services in the First Place? In the K8s cluster, our application code is running inside containers using the K8s Deployment resource. This deployment resource internally creates a K8s Pod resource, which in turn actually runs the containers. K8s assigns an IP address whenever a pod is created. The command below shows you an example IP address of the Pod Shell kubectl get pods <pod-name> -o wide Example Pod IP K8s pods are short-lived/ephemeral. So if a pod dies for any reason, K8s will automatically restart the pod, but the IP address assigned to that pod also changes. This in turn will lead to communication failure between the $Products_A$ and $Reviews_A$ microservice, as the IP address of $Reviews_A$ service is hardcoded in the configuration file of $Products_A$ service. This is problematic! Wouldn’t it be great if somehow a static IP address or a DNS name was assigned to the Pod? This will solve our problem. That’s where the Service resource of K8s comes to the rescue. I know the name Service sounds deceiving. A Service resource doesn’t run our containers. It merely provides and maintains a network identity for our Pod resource. This Service network identity provides both a static IP address and a DNS name. We just need to specify the ports on which the incoming request will be received and the port of the pod on which the request has to be sent. Services listen on a static address that doesn’t change and forwards the request to the unreliable container address. It internally keeps a map that always contains the latest IP address of the Pod. So, whenever a Pod restarts, it updates that map automatically so you get a seamless connection. Another benefit of using services is load balancing between replicas of the same application. With our imaginary application, because there are three instances of each microservice running on separate VMs, a service will load balance the request between the replicas of $Products_A$ $Proudcts_B$ $Products_C$. Load Balancing between Pods in K8s using Service Resource Until now, we have understood what are services and why are they needed. Let’s check out the different services that K8s offers and their use cases. How Do I make $Products_A$ Service Talk to $Reviews_A$ service? ClusterIP Service to the Rescue The following YAML defines a service of type ClusterIP YAML apiVersion: v1 kind: Service metadata: labels: app: reviews name: reviews spec: ports: - name: http port: 5000 protocol: TCP targetPort: 80 selector: app: reviews type: ClusterP The important fields in this YAML are targetPort: denotes the container port on which the request has to be forwarded port: denotes the port on which the service accepts incoming requests selector: used for associating/attaching a K8s Service resource to a K8s Pod resource. This is done by matching labels (key-value pairs) of the pod with selectors labels specified in the selector section. Selector labels must match to attach a service to a pod. The command below shows you the static IP address of the service which can be used by other Pods: Shell kubectl get service <service-name> You can use the above IP address for communication between the $Products_A$ and $Reviews_A$ microservices. There are two way to communicate using DNS. Pods residing in the same K8s Namespace can use the name of the service for DNS resolution Example → http://reviews:5000 Pods residing in other namespaces can use the following DNS notation → <service-name>.<namespace>.svc.cluster.local Let’s say $Products$ Pod resides in the products namespace and the $Reviews$ pod resides in reviews namespace. For the $Products$ pod to communicate with the $Reviews$ pod, you would use the following DNS → http://reviews.reviews.svc.cluster.local:5000 Now you can pass these addresses to your microservice for communication. Internal Communication Using Cluster IP Service Until now, we've understood how applications inside the K8s cluster talk to each other, but you made your application to be consumed by users on the internet. Let’s check out how it is done. How Do I Expose $Products_A$ Service Over the Internet? Every application that is deployed in K8s by default cannot be accessed over the network/internet. They need to be exposed Via a Service. K8s provides two Services to do just that: NodePort Service The following YAML can be used to create a NodePort service: YAML apiVersion: v1 kind: Service metadata: labels: app: reviews name: reviews spec: ports: - name: http port: 5000 protocol: TCP targetPort: 80 selector: app: reviews type: NodePort The above YAML definition is very similar to the ClusterIP Service definition. The only field that has changed is the type field from ClusterIP to NodePort. When you create a NodePort service, K8s will randomly select any available port on the node/VM between 30000 - 32767 and listen on it for incoming requests. You can specify the NodePort field yourself in the service definition, but you have to deal with the hassle of checking if the port is available on every node. After creating the service, to get the port on which this NodePort service is listening for requests, use the command: Shell kubectl get service <service-name> In our imaginary K8s cluster, this will happen on all the nodes: Node A (192.168.0.1:30519) Node B (192.168.0.2:30519) Node C (192.168.0.3:30519) You can access your application using the address nodeIP:30519. For Node A, it would be 192.168.0.1:30519. Exposing Application Using Nodeport Service But this is not practical. Users of your application won’t remember your IP address and port for accessing your app. They need a domain name like myapp.com to access your app. We solve this by putting an external load balancer in front of our VMs. The load balancer's public IP is mapped to myapp.com, so when you type myapp.com in a browser, it resolves to load balancer's IP. Then, the load balancer forwards these requests to any of our three nodes according to an algorithm configured in the load balancer. K8s provides another service to expose your application called the LoadBalancer service. LoadBalancer The following YAML can be used a create a LoadBalancer service: YAML apiVersion: v1 kind: Service metadata: labels: app: reviews name: reviews spec: ports: - name: http port: 5000 protocol: TCP targetPort: 80 selector: app: reviews type: LoadBalancer The above YAML definition is very similar to the ClusterIP Service definition. The only field that has changed is the type field from ClusterIP to LoadBalancer. The LoadBalancer service is an abstraction over the NodePort service. What’s special about this service is that, if you are using a managed Kubernetes service like EKS(AWS), GKS(GCP), AKS(Azure), or any other cloud provider, it provisions a load balancer by itself so that you don’t have the hassle setting up a load balancer yourself. After creating the service, to get the public IP address of the load balancer provisioned by this service, use the command: Shell kubectl get service <service-name> Load Balancer Service Example But what happens if you create a LoadBalancer service on your local K8s cluster or configured your own K8s from scratch in the cloud? The answer is: It won’t do anything. You will see a <Pending> state for the IP address section. The reason is that cloud-specific K8s clusters add other special programs/controllers that detect the LoadBalancer service creation and take the appropriate action accordingly. One thing to note here: The LoadBalancer service doesn’t expose any ports by itself like the NodePort service. LoadBalancer is a superset of NodePort, which is a superset of the clusterIP service, which means that, when you create a service of type NodePort, a ClusterIP service also gets created implicitly. So if you create a NodePort service for a $Products$ microservice, then this same service can be used for both internal communications by other pods and accessing the $Products$ service on the internet. Service Superset What Should We Use NodePort or LoadBalancer Service? If you read carefully, you will observe there is not much difference between the NodePort and LoadBalancer service, except for some automation being done in the latter one. Using a LoadBalancer service takes away the trivial task of configuring and provisioning LBs. Meanwhile, using a NodePort gives you the freedom to set up your own load-balancing solution, such that configuring environments that are not fully supported by Kubernetes, or even exposing one or more nodes’ IPs directly. So it depends upon the situation you are in, but the general rule of thumb is to try to use the LoadBalancer service first, but if it doesn’t work for your environment, go for the NodePort service That is it, these two services are used to expose your application outside of the cluster. But there is a slight problem. If you want to expose more than one application, you end up creating multiple NodePort / LoadBalancer Services. This gets the job done but you have to face some consequences because with each NodePort service you need to deal with the hassle of manually managing the IPs and ports. And with each LoadBalancer service, you go on increasing your cloud bill. No worries, K8s has addressed this issue with the use of Ingress and Ingress controllers. How to Expose Multiple Applications Without Creating Multiple Services Ingress and Ingress Controller Ingress and Ingress controllers are two different things. An Ingress controller is just another K8s deployment resource (an app running in a container), but what is special about this deployment is that it provides a single point of control for all incoming traffic into the Kubernetes cluster. Think of the Ingress controller as a smart proxy running in K8s and Ingress as a K8s resource that configures the routing logic of the Ingress controller. For people who have worked with Nginx, the Nginx webserver would be your Ingress controller, and the nginx.conf file would be your Ingress resource. With these, you can specify what request needs to be routed to which service in your K8s cluster on the basis of any request parameter, like host, URL path, subdomain, etc. The following YAML can be used to create an Ingress resource: YAML apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-wildcard-host spec: ingressClassName: nginx rules: - host: "products.myapp.com" http: paths: - pathType: Prefix path: "/products" backend: service: name: products-app port: number: 80 - host: "ratings.foo.com" http: paths: - pathType: Prefix path: "/ratings" backend: service: name: ratings-app port: number: 80 With the above YAML, we have exposed two services. If a request comes from products.myapp.com and the request path starts with /proudcts, we send it to our products-app K8s Service, which in turn forwards the request to our container. Similarly, if a request comes from ratings.bar.com it is forwarded to ratings-app Ingress YAML Explanation The important fields in these YAML are: ingressClassName: Usually, there is only one ingress controller in the cluster, but no one is stopping you from creating many. This field is used for the selection of ingress controller, similar to the selector field of services service: denotes the name of the service on which the request has to be forwarded An ingress resource should be created in the same namespace where the corresponding K8s Service resides, but an ingress controller can be placed in any namespace. It will automatically detect Ingresess defined in other namespaces on the basis of ingressClassName. An important thing to remember is that an Ingress by itself doesn’t expose any port. In the end, it’s just a K8s deployment resource. You need a service(NodePort/LoadBalancer) in front of it to expose it. Exposing Application Using Load Balancer Service Conclusion Congratulations on sticking around until the end. Let’s summarize what we have learned so far. Kubernetes gives us three types of Service Resource Object Cluster IP Service→ Used for internal communication among workloads and handles Service Discovery. NodePort Service→ Used for exposing applications over the internet, mostly used in development environments LoadBalancer Service→ Used for exposing applications over the internet Provision actual load balancer on the cloud, on supported cloud platforms Ingress Resource→ Used for controlling incoming traffic in the Kubernetes cluster Various implementations are available each with its pros and cons That’s it for this blog post. If you liked this blog post, you can follow me on Twitter @SharadRegoti, where I talk about Microservices, Cloud, Kubernetes, and Golang.