Data Engineering Resources

How to Read a Large CSV File With Java 8 and Stream API

Scenario: you have to parse a large CSV file (~90MB), practically read the file, and create one java object for each of the lines. What do you do?

September 28, 2016

by Eugen Hoble

· 264,400 Views · 27 Likes

How to Quickly Load 380K Items Into MySQL

See how a bit of legwork and a single statement can save you a lot of time and overhead when working with MySQL data.

September 27, 2016

by Eugen Hoble

· 11,145 Views · 6 Likes

IoT Push Notifications: Arduino, Firebase, and Android

This quick and easy tutorial will show you how to implement push notifications for IoT apps.

September 27, 2016

by Francesco Azzola

· 35,629 Views · 9 Likes

IOPS: Benchmarking Disk I/O – AWS vs DigitalOcean

In this article, we will see how to do this and we will compare the performance between two different providers.

September 26, 2016

by Christopher Sam K

· 37,694 Views · 6 Likes

AWS and Azure: SLAs and Service Credits

See how both AWS and Microsoft Azure handle breaches to their SLAs with this head-to-head comparison. Learn how each rewards users if their service goes down.

September 26, 2016

by Christopher Sam K

· 22,595 Views · 2 Likes

How to Authenticate With JGit

In this post, Alex Soto shows you how to authenticate to a Git repository with JGit.

September 26, 2016

by Alex Soto

· 32,456 Views · 6 Likes

Head to Head: Apache Impala vs. Amazon Redshift

See up-close comparisons of Impala and Redshift, complete with benchmarks and feature comparisons.

September 23, 2016

by Mostafa Mokhtar

· 10,592 Views · 2 Likes

How DBAs Should Be Evolving With Database Automation

As DBAs are saddled with more duties and responsibilities, it can be a strain to handle them manually. See where automation can make your life easier.

September 23, 2016

by Yaniv Yehuda

· 11,308 Views · 4 Likes

''AAA'' API Testing - Accessible, Affordable, Agile

How Service Virtualization addresses the growing challenge of testing third-party APIs.

September 22, 2016

by Heather Peyton

· 5,893 Views · 4 Likes

Map DynamoDB Items to Objects Using DynamoDBMapper

See a demonstration of the DynamoDBMapper class, which helps map your tables' items to Java objects.

September 22, 2016

by Emmanouil Gkatziouras

CORE

· 18,771 Views · 4 Likes

MongoDB Point-in-Time Backups Made Easy

See how to protect your data on MongoDB with point-in-time backups. This article walks you through the steps to configure and use your backups to ensure your data's safety.

September 22, 2016

by David Murphy

· 10,518 Views · 2 Likes

Inside CockroachDB’s Survivability Model

This interview with Marc Berhault takes a look into CockroachDB and its heavy focus on data redundancy. This model allows CockroachDB to survive and thrive even after heavy hits.

September 21, 2016

by Dave Avery

· 5,898 Views · 1 Like

Serenity BDD and the Screenplay Pattern: Designing SOLID Actors

In this article, Jan Molak answers some of the most popular questions related to designing Screenplay Actors and making them SOLID.

September 21, 2016

by Jan Molak

· 8,252 Views · 5 Likes

Modeling Microservices at Spotify with Petter Mahlen

At a microservices talk in March in Sweden, Petter Mahlen, Backend Infrastructure Engineer at Spotify, spoke to a packed house at Jfokus about microservices.

September 21, 2016

by Carissa Karcher

· 23,179 Views · 15 Likes

Change Data Capture Using Apache NiFi

Capturing all changes from a relational database with Apache NiFi is very easy and explained. CDC is a common use case for extracting transactional data in a streaming manner to populate a datawarehouse or datalake in Hadoop.

September 20, 2016

by Mark Herring

· 20,844 Views · 4 Likes

A Couchbase Index Technique for LIKE Predicates With Wildcard

Take a look at some of the latest changes to Couchbase in action, including a more efficient way to index and query your data.

September 20, 2016

by Tai Tran

· 11,267 Views · 13 Likes

Imperative vs. Declarative Query Languages: What's the Difference?

To say that any non-imperative query language is declarative is too broad a definition. See how declarative and imperative languages work and where the work best.

September 20, 2016

by Joy Chao

· 14,957 Views · 4 Likes

How MySQL X Plugin Works Under the Hood

X Plugin allows MySQL to function as a document store. We don’t need to define any schema or use SQL language while still being a fully ACID database.

September 20, 2016

by Miguel Angel Nieto

· 6,787 Views · 3 Likes

OpenStack: Core Components

OpenStack is an open source cloud software which consists of a series of allied projects controlling large pools of computing, storage, and network resources in a data center while managing through a dashboard. Let's look in detail at its various core components.

Updated September 19, 2016

by Sree Tejaswi

· 75,345 Views · 22 Likes

High-Quality Recommendation Systems with Elasticsearch: Part I

We argue that recommendations and search are two sides of the same coin. Both rank content for a user based on “relevance.” The only difference is whether a keyword query is provided. Read this article to find out more.

Updated September 18, 2016

by Doug Turnbull

· 11,964 Views · 7 Likes

The Latest Data Engineering Topics