DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
  1. DZone
  2. Data Engineering
  3. Data
  4. The Cost of Ignorance: Why You Need a Chain of Evidence, Not Just Logs

The Cost of Ignorance: Why You Need a Chain of Evidence, Not Just Logs

In your applications, especially a distributed environment, you'll need to have a record of what goes on, for security, performance, and other purposes.

Michael Bissell user avatar by
Michael Bissell
·
Sep. 14, 18 · Opinion
Like (4)
Save
Tweet
Share
3.63K Views

Join the DZone community and get the full member experience.

Join For Free

You may have seen my posting on East/West is the New North/South. The bottom line is that traditional API Gateway models simply don't provide the level of security we need in modern microservice architecture. The problem is that only 20% of the traffic (that is the inbound traffic up until the gateway) is secure, everything inside the data center is "trusted."

Sure, it may be secured by two-way certs, IP whitelisting, or other network models, but there is no identity. We don't know WHY "Service A" called to "Service B." We might not even know that "Service A" was what called to "Service B" just that something made a call.

But why do we care? We trust "Service A" don't we?

In Infrastructure, Legal and Embarrassment -- Why We Secure Our Systems, we looked at the three reasons we care. We looked at things we can do to lock down those services, and what we mean by "fine-grained access control." And then, in an offhand way, I said "log everything" which is a gross over-simplification.

You see simple logs aren't enough. If you have a breach, you're going to need to provide a complete record from what kicked off the original request, what other requests it kicked off, and what data was exposed.

Let's look at our three reasons we secure data, and what we need to know about those transactions:

Infrastructure — Root Cause Analysis Paralysis

Even if you have your services secure, if you don't know why "Service A" called to "Service B," you have a lot of trouble determining your RCA (root cause of the problem). I think most of us have been forced to answer, "well it stopped so I think we're okay but keep an eye on it." If you're lucky the problem doesn't come back, but relying on luck isn't really a "best practice."

If you can look at the logs, determine which transactions are suspicious, and then drill your way back up the chain to see that a very special user is able to start a cascading series of requests in just such a way, you can close that door. Without the upstream requests that caused Service A to fire off an odd request, you may never know who was responsible and never be able to adequately secure the environment.

Legal — Just How Bad Is the Problem?

You may feel you're in an industry not covered by the new California Privacy laws or by Europe's GDPR or even State of New York's Financial Services laws, but honestly, if you deal with money you probably are. And then there's HIPAA...

All of these require an immutable chain of evidence. Note that word "immutable." This is where syslogs aren't enough. Not only do you need to know User A called App B that called to Service A that needed to make a callout to Services B and C, but you can't just show a few lines of logfiles stitching those requests together. You have to be able to show that ever request was signed and that the record was not altered.

There aren't a lot of ways to do that unless you're registering each application with its own set of keys, and using those keys to send and receive data on each and every transaction. It's not about trusted and untrusted services anymore, it's about an untrusted logging system now because, you know, lawyers.

The other thing is that we get charged per exposure. The average cost per exposure varies anywhere from $110 to $336 per record, but if you can't determine exactly what was exposed, you could be on the hook for your entire database...

Cost per exposure of personal data by industry

Health

$336

 

Communications

$228

Financial

$274

 

Energy

$196

Services

$264

 

Consumer

$177

Life science

$259

 

Retail

$144

Industrial

$251

 

Hospitality/Entertainment

$131

Technology

$245

 

Research

$123

Education

$240

 

Public sector

$110

Transportation

$239

 



So even at $123 per record in the "Research" business, a Cambridge Analytica breach of 87 million records could have cost over $10 billion in fines.

Business — We Demand Answers

The Cambridge Analytica exposure wasn't a breach so it wasn't something that could be fined, but it definitely put Facebook in a tough place, and part of the reason it was tough was because they couldn't definitively answer what seemed like a simple question: "How many records were exposed?"

This stuff is complicated, it's hard enough talking to the DevOps team about the problem, but talking to your shareholders, a news outlet or lawmakers makes it even harder. Saying, "we don't really know what data was exposed" makes you appear to either be lying or incompetent. Either way, it's not a good day.

So to sum up, service to service security isn't just about securing the flow, it's about creating a full chain of evidence so your DevOps team can fix problems, you can limit your legal exposure, and you can simplify business explanations when something goes wrong.

Data (computing) microservice

Published at DZone with permission of Michael Bissell, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Implementing Infinite Scroll in jOOQ
  • Promises, Thenables, and Lazy-Evaluation: What, Why, How
  • What Should You Know About Graph Database’s Scalability?
  • Hackerman [Comic]

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: