When you discover a security vulnerability affecting your environment, you want to fix it. Quick.
But, as you’re probably painfully aware, it’s not always that simple. Many of us have already come across vulnerabilities that can’t be completely removed—leaving even the most experienced security pros feeling severely challenged. This year’s Verizon Data Breach Investigation Report addressed this type of problem, and we felt it was worth opening up a broader conversation around this unfortunate reality.
No matter what stage you’re at—on-premise, hybrid, or fully in the cloud—here’s what to do about those stubborn vulnerabilities you can’t get rid of.
What Do You Mean You Can’t Fix It?
The practice of vulnerability management is, by nature, cyclical and ongoing. There’s no such thing as being “done” with managing vulnerabilities since new ones crop up every day, no matter how well a security program is designed or how diligently systems are updated.
Vulnerabilities can come from a variety of sources. Sometimes development teams (eager to get the job done) will circumvent the chain of command and install unauthorized packages in the base AMI or even manually on production environments. It’s pretty tough for security teams to verify the attack surface of these types of packages if… they don’t know they exist.
These human hurdles to vulnerability management can certainly make the job of a security pro harder, at least without the right tools in place.
The good news is this “blind spot” can be corrected using a vulnerability management platform like Threat Stack, which automatically examines package information on every workload and tells the user whether there are any vulnerable packages. This can keep rogue users from poking holes in your otherwise watertight boat.
But, as you may know, some vulnerabilities simply can’t be fixed. This could be due to a business process that can’t be changed, the lack of an available patch, or basic incompatibility. For example, vulnerabilities can come about due to persistent and widespread bugs (e.g. Heartbleed for SSL or Shellshock in the Unix Bash shell.) Generally, these bugs can be patched once discovered. But, depending on their complexity and how widespread they are, it can take time. And in some cases, there’s no ready solution because vendors and security people don’t agree on a patch or the speed at which it should be released, as in the case of an Android Wi-Fi Direct vulnerability that caused some Android devices to reboot under a remote attacker's orders.
So what should you do when you’re staring at a vulnerability that you can’t get rid of?
Keep Your Friends Close, and Your Vulnerabilities Closer
Simply put, if you can’t beat ‘em, join ‘em. Or, perhaps more accurately in the world of security, if you can’t eliminate, mitigate. Sometimes it’s your only option, so you might as well do it right.
Here’s what that looks like in practice:
If one of your systems can’t be patched or updated, you should first identify the vulnerable system. Then you should change the configuration settings to mitigate the risk. A significant example that comes to mind is the recent RCE (remote command execution) vulnerability that surfaced in the ImageMagick application. In the absence of an initial fix, the ImageTragick disclosure site advised users to make configuration changes to mitigate risk.
Next, if possible, isolate the vulnerable system so it cannot talk directly to other parts of the system (thus decreasing the potential attack surface). In the case of compromised data, for example, you could restrict an affected node from communicating outbound to the Internet so that, in case of a breach, it can’t send data out of the environment. Or if this is a system that was previously accessible from the public Internet, you could place it behind a VPN (virtual private network) in order to restrict inbound connections to a trusted group of users.
Yes, we said you’re just going to have to live with the vulnerability, but that’s only until you can replace its source. If the affected system is inoperable, have a plan for taking it out of circulation in a manner that avoids serious disruption to your business. It may take some time to secure the budget, find a new system that can take over, or migrate the data, but in the long run it can be worth it to remove systems that can’t be fixed in order to mitigate serious vulnerabilities. This is where designing your application to run in cloud environments has is benefits, and the discussion of treating systems like cattle and not pets comes into practical usage.
We saved the best (and, in our opinion, most important) for last. Vulnerability management is the best way to identify new devices and services as they are added to your environment (with or without authorization). The more you know, the better you can protect yourself. It’s a good idea to review changes from scan-to-scan in order to zero in on unfamiliar devices that have been added or any other deviations from the standard configurations, paying particular attention to systems that have known vulnerabilities in them.
A vulnerability management tool like Threat Stack can help you automate and accelerate this process so no one has to comb through system changes manually looking for suspicious deviations.
If You Can’t Fix It, Mitigate It
It’s an unfortunate reality of cloud security that, even when we know what the “right way” is, we can’t always execute on it. That can be a hard pill to swallow, but it helps us approach vulnerabilities with a realistic mindset. First, try to eliminate them. If you can’t do that, mitigate them to the best of your ability and then get on with it. It’s part of the job of a security pro to know when it’s as secure as it’s going to get.