Data Engineering Resources

Improving Performance With SQL Aggregate Functions

SQL aggregate functions helped me take the performance of my backend application to the next level and avoid the bottlenecks that were slowing it down.

January 5, 2022

by Antonello Zanini

· 5,149 Views · 2 Likes

How To Create a MongoDB Projection?

In this post, we will learn how to create a MongoDB Projection. Basically, projections allow us to retrieve selective fields of a document while querying a collection.

January 5, 2022

by Saurabh Dashora

CORE

· 7,034 Views · 3 Likes

Kubeflow Fundamentals Part 6: Working With Jupyter Lab Notebooks

In this post, we’ll focus on getting more familiar with Jupyter notebooks and how we can leverage them within Kubeflow as part of our machine learning workflow.

January 5, 2022

by Jimmy Guerrero

· 5,283 Views · 3 Likes

Simplify Migrating From Kafka to Pulsar With Kafka Connect Support

In this article, we will discuss streaming sources that push data into Pulsar from another system and sinks that send data from Pulsar to another destination.

January 5, 2022

by Enrico Olivelli

· 5,542 Views · 3 Likes

The “Onion Peel” Approach to Hyper Intelligent Automation

Let's see how the Onion bulb scales represent various stages in the journey of Intelligent Automation that an enterprise can look to undertake systematically.

January 5, 2022

by Ved Prakash

· 6,317 Views · 4 Likes

CockroachDB CDC Using Minio as Cloud Storage Sink - Part 3

This is the third post in the series and today, we’re going to explore CDC capability in CockroachDB Enterprise Edition using Minio object store as a sink.

January 5, 2022

by Artem Ervits

CORE

· 6,218 Views · 3 Likes

JConf Peru 2021 Conference Report

At JConf Peru 2021, a regional Java community conference, I delivered three talks focused on Java/Jakarta EE and Azure.

January 4, 2022

by Reza Rahman

· 3,860 Views · 4 Likes

CockroachDB CDC With Hadoop Ozone S3 Gateway and Docker Compose - Part 4

This is the fourth tutorial post on CockroachDB and Docker Compose. Today, we'll evaluate the Hadoop Ozone object store for CockroachDB object-store sink viability.

January 4, 2022

by Artem Ervits

CORE

· 5,266 Views · 3 Likes

The Ultimate Code Quality Checklist

Code review is a complicated process, and every company must follow a code review checklist before conducting one.

January 4, 2022

by Alex Duecode

· 9,563 Views · 10 Likes

Distributing a Java Command-Line Application

In this article, I'll show you how to package and distribute a java command-line application and the challenges one has to face to implement it.

January 4, 2022

by Loïc Rouchon

· 11,252 Views · 7 Likes

Apache Kafka as Cloud-Native iPaaS Integration Middleware

This post explores why Apache Kafka is the new black for integration projects, how it fits cloud-native iPaaS solutions, and why event streaming is a new software category.

January 4, 2022

by Kai Wähner

CORE

· 6,263 Views · 5 Likes

Snowflake Data Processing With Snowpark DataFrames

Snowpark is a new developer library in Snowflake that provides an API to process data using programming languages like Scala (and later on Java or Python), instead of SQL.

January 4, 2022

by Istvan Szegedi

· 15,707 Views · 5 Likes

Micro-Frontend Architecture

The goal of this architecture is to see web applications as a composition of functionalities where each one is worked by independent teams.

January 4, 2022

by Arnau Gris

· 9,053 Views · 15 Likes

Three Easy Ways to Run Kafka Without Zookeeper

Since version 2.8, we now can run a Kafka cluster without Zookeeper. This article will go over three easy ways to get started with a single node cluster using containers.

January 4, 2022

by Hugo Guerrero

CORE

· 7,552 Views · 4 Likes

The Role of AR/VR in Transforming the Manufacturing Industry Digitally

In this article, find out how the manufacturing industry is adopting AR and VR to transform the way it operates, and therefore overcoming several challenges.

January 4, 2022

by Sandeep Gundla

· 4,334 Views · 2 Likes

How to Generate Customer Success Analytics in Snowflake

Here's a quick demonstration on how to generate quick insights on customer success data in Snowflake using the platform Datameer.

January 3, 2022

by Jason Scharf

· 5,263 Views · 5 Likes

Everything You Need to Know About AWS Graviton2

Here's everything you need to know about AWS Graviton2, including how it compares to other processors and what migrating your workload entails.

January 3, 2022

by Ned Hallett

· 5,276 Views · 4 Likes

How To Utilize NoSQL DB: Graph Database Examples

With the rising popularity of NoSQL databases among developers, read this post to explore their possibilities, with the focus on Graph Database utilization.

January 3, 2022

by Tetiana Stoyko

· 4,101 Views · 2 Likes

SerpApi YouTube Data Extraction Tool

YouTube data has become a major part of machine learning and data analytics. Here's how to use SerpApi to extract YouTube data and query it for analysis.

January 3, 2022

by Charles Mabwa

· 1,485 Views · 3 Likes

Chef Inspec Configuration Steps

Chef InSpec is an open-source testing framework used to test or validate configurations. Here's how to configure Chef Inspec in your local system.

January 3, 2022

by Aki Ramanjaneyulu

· 8,245 Views · 2 Likes

The Latest Data Engineering Topics