How to Cut Through Vendor Claims and Marketing Hype When Evaluating New Security Tools
Cut through the bull when selecting a new security tool.
Join the DZone community and get the full member experience.Join For Free
As we've pointed out in a couple of recent articles, Machine Learning (ML) has been billed as a savior for short-staffed security teams — a silver bullet that can single-handedly identify and mitigate every security threat automatically. As we usually do with silver bullet solutions, we've cautioned readers to distinguish between the hype and reality. While ML has many strengths and is here to stay, it's only a part of the solution in the world of cybersecurity — not the solution itself. Human input is still essential to draw meaningful conclusions and define appropriate action.
In today's post, we're continuing to advise readers that it's essential to go below the surface, to distinguish between the hype and reality when evaluating a cybersecurity solution. Remember: a beautiful package may open up to reveal a beautiful can of worms. Keep your eyes open, investigate below the surface, and avoid nasty surprises.
Abstraction permeates the computing field, including security. Each abstraction layer makes the problem easier to conceptualize and reason about, but it also hides details and can make investigating deviances from the abstraction difficult. Machine Learning (ML) represents a powerful abstraction of programming via statistics and patterns. Like other abstractions, it makes the problem easier to think about: "The computer is learning how to handle this data." Also like other abstractions, it makes deviances hard to debug: "Why did we miss that one?" Even if users have the propensity to investigate into the model, proprietary methods often prevent peering behind the curtain and hand tuning models.
We live in a world that tries to give more and more of our agency to computers. We curate our feeds with thumbs-up or thumbs-down, and hopefully, that improves the content. Our attempts to tune the machine learning algorithm do not always bear fruit. Sometimes, I still get horrible songs on Pandora. Facebook still shows me things I do not care about. I've only tuned Pinterest a little, so it shows me all sorts of weird things. Sometimes it's funny, sometimes it's frustrating. In a security setting, however, it can be dangerous.
Take a moment to consider the motivations behind looking for a security solution. Now consider how vendors sell their products. Do those two align? Unfortunately, the money grab for funding leads to flashy demos and hyperbole, both from vendors and from internal stakeholders. The necessity of this presentation is nearly a fait accompli, but flashy demos and hyperbole are often not the goals for a security solution. This means that a lot of time and effort goes into making a fancy cover. Defenders forget the saying: "Don't judge a book by its cover," and this can lead unscrupulous advocates of security solutions to spend inordinate amounts of time on the cover compared to the substance inside.
Enumerate some of that substance a defender wants from a security solution:
- Detect security incidents
- Achieve compliance
- Provide visibility into systems
- Support infrastructure in transition
- Operate at speed and scale
- Protect past, present, and future innovation
A demo can and should illustrate the capabilities mentioned above, but it can also sweep weaknesses under the rug. Also, note that the various items listed above do not include the "how" of the solutions. Evaluating the "how" can help us make sure the solution adequately deals with the "what," but it does not supersede the "what." Whether a solution uses machine learning or the blockchain becomes irrelevant when evaluating whether it meets the needs of the user.
We find that while customers appreciate a sleek UI, they find the most value in the data you provide. With that in mind, be sure that your security tools are data driven, not marketing driven.
Published at DZone with permission of Nathan Cooprider, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.