Analyze VPC Flow Logs for Security and Performance

The logs capture important information about the IP traffic to and from network interfaces, subnets and VPCs in the AWS infrastructure. They are used to monitor security by tracking traffic reaching and leaving the resources(instance, databases, etc.) in the AWS infrastructure.

The logs provide network flow visibility in the dynamic cloud which helps in tightening security and understand network performance. 

And of course, keep the SecOps team off your back.

Security

The flow logs provide a number of avenues to tighten your security and make sure everything is connecting as needed.

1. They can be used to boost data collection to help correlate and strengthen security insights.
2. Help you create baselines of your standardized network activities which in turn helps in identifying abnormal events.
3. Identifying potential botnet activity is easier by comparing time-stamps and periodicity of certain traffic. You can also directly look for known botnet connections to interfaces and block them.
4. Identify and block vulnerability scans by checking for ping sweeps, port scans and other malicious activities associated to discover weaknesses in the network.
5. You can also improve troubleshooting of performance problems.
6. Optimize connectivity inside your organization for developers, testers and, ITOps teams. 

Performance

The VPC flow logs help in identifying latencies, establish performance baselines, and tweak apps as needed.

It provides granular information like traffic flow duration and latencies and bytes sent, which in turn helps in identifying performance issues quickly. Having quantifiable data from the logs helps in providing a better user experience. 

VPC flow logs are employed to monitor internal application services. The applications build on the microservices architecture heavily rely on internal traffic to communicate.

Note: Understanding VPC flow logs also provides an opportunity to save bandwidth costs like Mikuláš did. You can read more about it here.

Analyzing VPC Flow Logs

Enabling them is the easy part but analyzing VPC flow logs is just another ball game. There are multiple ways of analyzing the VPC flow logs but most of them are expensive in terms of bandwidth and time needed to set them up. Below are certain ways to set up your dashboard.

• Analyzing VPC Flow Logs with Amazon Kinesis Firehose, Amazon Athena, and Amazon QuickSight

• Analyzing AWS VPC Flow Logs using Apache Parquet Files and Amazon Athena

Conclusion

AWS has provided these logs to improve network visibility and tackle security, performance, and cost.

P.S. – We recently opened the registrations to the closed beta for visualizing VPC flow logs in a virtual environment.

Visual AWS VPC flow