Over a million developers have joined DZone.
{{announcement.body}}
{{announcement.title}}

GDPR Compliant Forget-Me App With Spring Integration (Part 1): Requirements

DZone's Guide to

GDPR Compliant Forget-Me App With Spring Integration (Part 1): Requirements

Read this article in order to learn more about the requirements for an open-source forget-me app in compliance with GDPR.

· Integration Zone ·
Free Resource

The new Gartner Critical Capabilities report explains how APIs and microservices enable digital leaders to deliver better B2B, open banking and mobile projects.

In preparation for the enforcement of GDPR which becomes final on the 25th of May 2018, I’m creating a simple, open-source forget-me app for springuni.com in compliance with Art. 17 GDPR (‘right to be forgotten’). In the first phrase, I’d like to support a simple forget-me process driven by the Scatter-Gather messaging pattern using Spring Integration and RabbitMQ, although I’ve got a ton of ideas about making a generally usable, granular, and flexible consent management app for fellow bloggers. Before going into coding the app instantly, in this article, I just set the initial requirements for myself and for starting a discussion if someone else is interested in having their own forget-me app.

Non-Technical Requirements

  1. GDPR is tricky in the sense that you have to delete folks’ personal data from all your systems, but you still have to be able to prove that you did comply with their request. Nowadays, one's email address is the primary handle used to identify individuals and 90% of my subscribers use Gmail, 5% use Yahoo Mail, and the rest use other providers. Perhaps they are self-hosted. Having said that, the first requirement is to use social login backed by Google’s, Yahoo’s, and Facebook’s OAuth2-secured authentication flow. The last one would probably allow those people to login in, who use a customer email address of some kind. Spring Security 5’s new OAuth2 support will be just fine for the purpose.
  2. The app cannot persist personal data unless it’s absolutely necessary to carry the forget-me process out. Yet, it has to be able to state that the request of a certain individual has been successfully completed. By hashing (probably an SHA-2 variant will do) email addresses and creating a user profile without any personal details this requirement can be satisfied.
  3. Let’s think about the life-cycle of a subscriber’s subscription. Someone signs up for my newsletter, they may unsubscribe and eventually they may ask me to forget all their personal data stored. Given these three uses cases, a subscriber can be in the following states.
    • SUBSCRIBED — denotes an active subscriber who gave their consent to receive newsletters.
    • UNSUBSCRIBED — denotes an inactive subscriber, who withdrew their consent to receive newsletters.
    • FORGET_PENDING — means that they requested that their personal data be erased with regards to a subscription
    • FORGET_COMPLETED — means that personal data of a subscription has been deleted
    • FORGET_FAILED — means that personal data couldn’t be deleted due to a technical issues

  1. When someone signs up to my newsletter, the forget-me app should be notified and a subscriber and a subscription with status SUBSCRIBED needs to be created there, the same applies when for unsubscribes.
  2. Upon request, all systems handling personal data needs to be asked to remove them.
  3. When someone would try to log in without being having a subscription, that request can be rejected.

UI Design

Eventually, these requirements boil down to a simple UI with two pages. As a back-end guy, I just prefer to do the mock-ups quickly without any fuss on a piece of paper. Yes, I know there are tons of wireframing tools but nothing as fast as drawing by hand with a pencil.

After logging in, subscribers can see the current status of their subscriptions and a change log.

Provided that the currently logged in user is in a(n) (UN)SUBSCRIBED state, the Forget Me button becomes active and after a confirmation, the process can be started.

Technical Requirements

Even before getting into the technical requirements, I’d like to make a mental note to myself that I’m not allowed to spend more than 8 hours to complete a deployable prototype based on Spring Boot 2 to Heroku.

Software Stack

  1. I don’t expect heavy traffic, so it’s perfectly okay and it’s rather desirable that the app go into sleep mode after being inactive for a while. Heroku makes a Dyno sleep after 30 mins of inactivity.
  2. I’ve got only a single requirement for the database: keep running costs low. I chose PostgreSQL for the prototype as Heroku provides that out of the box and other major cloud providers (eg. Amazon RDS for PostgreSQL and Google Cloud SQL for PostgreSQL) support it as well.
  3. Putting a UI together is always challenging, as there are so many JS frameworks out there. To keep things simple, this is going to be a standard Spring MVC app with Thymeleaf as the template engine and VueJS to handle UI updates.
  4. The app will be connected to message broker receiving subscriber update messages and emitting erasure requests. Heroku supports various RabbitMQ providers, I used CloudAMQP before, probably will stick to that.
  5. Spring Integration will be used to define and execute message flows
  6. In the first phrase, only a single system MailerLite will be supported, but I design a plugin-able architecture to be able to add other providers easily

Message Flows

There will be two distinct messages flows, one for receiving subscriber updates from other systems from through a webhook and the other one which handles deletion requests.

In the first case an external system submits a piece of JSON describing a SUBSCRIBE or an UNSUBSCRIBE event and upon receiving that, the forget-me app registers a new subscriber and a subscription specific to that data handler. In case of MailerLite, the payload looks like this. Messages transformers will be used to translate the provider-specific incoming data and extract the email address and the event’s type.

Upon requesting data erasure, the subscriber’s request will be put into a message queue and that request will be broadcast to all the registered data handlers. Thereafter they will reply back with either an ACK or a NACK and in both cases, the result will be recorded.

Messages Exchanged

Note: highlighted fields are messages headers, they’re just displayed together with the message body for brevity.

Webhook

As I mentioned above, data provided through the webhook varies widely, but with provider-specific message transformers, they’re going to be transformed to the following generic message.

{
   "event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
   "event_time":1526892561,
   "event_type":"webhook",
   "data_handler_name":"mailerlite",
   "data_handler_id":"5d4ae976-e397-4e62-9458-2a0f51c8c988",
   "subscriber_email":"test@springuni.com",
   "subscriber_status":"SUBSCRIBED"
}

Forget request

{
   "event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
   "event_time":1526892561,
   "event_type":"forget-request",
   "data_handler_name":"mailerlite",
   "subscription_id":"99d2a1a3-84d6-48fd-8ef9-e4681f743958",
   "subscriber_email":"test@springuni.com"
}

Forget response

{
   "event_id":"294cab6a-d915-4654-9fc9-afc6410bf62c",
   "event_time":1526892561,
   "event_type":"forget-response",
   "data_handler_name":"mailerlite",
   "subscription_id":"99d2a1a3-84d6-48fd-8ef9-e4681f743958",
   "acknowledged":true
}

REST API

The app is going to have a REST API with which the UI will communicate and also it will be used for performing administrative tasks as well.

Method Endpoint Access level
GET /api/subscribers admin
POST /api/subscribers admin
GET /api/subscribers/{subscriber_id} admin
POST /api/subscribers/{subscriber_id}/forget admin
GET /api/subscribers/me user
POST /api/subscribers/me/forget user
GET /api/datahandlers admin
POST /api/datahandlers admin
POST /api/datahandlers/{datahandler_id}/resetkey admin
POST /webhook/{datahandler_id}/{datahandler_key} public

The first phrase on the endpoints in italics will be implemented.

Project’s Resources

The forget-me app has been open sourced under AGPL v3 and it’s published to GitHub. Should you encounter an issue or have an idea, submit a ticket to JIRA.

Next in This series:

GDPR Compliant Forget-Me App With Spring Integration (Part 2): In and Outbound Messaging 

This second part focuses on how to define in- and outbound messaging with Spring Integration’s AMQP support by using the new Java DSL, which is now (as of version 5) part of the core project and doesn’t have to be included as a separate dependency.

GDPR Compliant Forget-Me App With Spring Integration (Part 3): Registering Message Handler Dynamically 

Although there will be only a single adapter supporting MailerLite, the app is modular in design and adapters are activated dynamically once they’re configured.

GDPR Compliant Forget-Me App With Spring Integration (Part 3): Oauth2 Login 

Spring Security 5’s brand new OAuth2 support is used and in this part, we’ll look into how the login flow can be customized.

The new Gartner Critical Capabilities for Full Lifecycle API Management report shows how CA Technologies helps digital leaders with their B2B, open banking, and mobile initiatives. Get your copy from CA Technologies.

Topics:
spring integration ,enterprise integration patterns ,gdpr ,requirements analysis ,rabbitmq ,spring mvc ,integration

Published at DZone with permission of

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}