Big data has been a force to be reckoned with. It has given brands tremendous opportunities to slash expenses and better engage with their customers. However, as with all technological revolutions, regulators haven’t been far behind.
Chris Forsyth, Former Partner at Freshfields Bruckhaus Deringer LLP, said it has created a double-edged sword for developers.
“Big data creates new opportunities for more effective marketing, but also creates a compliance burden and business risk.”
All professionals working with big data must have a clear understanding of privacy laws and take all reasonable efforts to comply with them. These laws are particularly stringent in the EU, which can create additional challenges for American and Asian brands expanding there.
Developers may need to work around other legal quandaries, such as if users are concealing their actual location with a VPN Proxy or Tor. However, there is legal precedent to argue that the laws of the country where a server is located would apply, which could mean brands don’t need to trace the user’s actual location.
Microsoft, Consumer Essentials, Nordstrom and many other brands are training their teams to better understand EU privacy laws.
EU Privacy Laws Affecting Big Data Engineers and Developers
The European Union has been very strict about protecting people’s privacy. They have passed several privacy laws that have significant issues for big data engineers. A few are highlighted below.
Right to Be Forgotten
The Right to Be Forgotten Law was passed in 2014. This law was intended to safeguard privacy of people after their pictures or names have been presented in an unflattering light.
Data Protection Directive
The Data Protection Directive was enacted in 1995. It prohibited the collection of data for unnecessary purposes.
General Data Protection Regulation
The European Commission introduced a new bill in 2012 to modernize the Data Protection Directive. While the preceding law would still take effect, the new law would include new changes, including:
“A harmonized pan-EU regulation, replacing the existing patchwork of 27 national regulations; an improvement of the current system of binding corporate rules for a save transfer of data outside the EU and a regime allowing better control over individual’s data.”
How Big Data Experts Can Comply With these Regulations
While some of these regulations have been in effect for years, compliance is still a learning curve in the age of big data. New policies are being implemented.
Big data scientists and developers must take the following measures to avoid regulatory breaches.
Rely More Heavily On Hive and Hadoop as Geolocation Solutions
While the EU intends to create a more uniform framework for safeguarding the privacy of its citizens, laws may still among member nations. Privacy laws will also vary outside the EU.
Using geolocation technology will be more important than ever after the General Data Protection Regulation is enacted. Here are some ways brands will need to use HGridand other big data tools to analyze geospatial data:
- They can use these tools to identify the user’s location, so they know which jurisdiction’s privacy laws apply.
- They can customize the browsing experience for users according to the laws of their region.
Use More Versatile Big Data Extraction Tools to Comply With the Right to Be Forgotten Law
Over the past two years, publishers have received over one million requests to remove links under the Right to Be Forgotten Law. While this is relatively easy for small publishers, it’s a compliance nightmare for brands relying on big data. They must use state-of-the-art big data extraction tools to extract and delete the data within the timeframe allowed under the law.
Tools like Hadoop will bemore important than ever. They allow administrators to access and delete any dataset with minimal challenges.
Create Narrower Data Sets
Brands in the United States are very liberal with data collection, because they have much more leeway. Under the Data Protection Directive, they must limit the scope much more carefully. For example, brands selling cable modems may be scrutinized for collecting customer credit rating information.
Many data tracking tools don’t discriminate with the data they collect. Developers and data engineers have to be careful not to unwittingly collect unnecessary data, lest they run afowl with the law.
The easiest solution is to limit the range of their datasets. They must be structured so any accidentally collected data that is irrelevant is unretainable. All fields need to be carefully scrutinized and brands need to make a good-faith argument why they are included.