Data Engineering Resources

Build a Data Analytics Platform With Flask, SQL, and Redis

Building a Flask-based web app that has dynamic querying for population thresholds, Redis caching for faster queries, and secure, scalable architecture.

February 17, 2025

by Sushma Kukkadapu

· 5,208 Views · 2 Likes

From Data to Decisions: Visualizing SAP Insights With Python

SAP data insights simplified with Python: A versatile tool for advanced analytics, automation, and managing large datasets.

February 17, 2025

by Prasanna Chandran Melnatami Krishnaram

· 2,659 Views · 5 Likes

Why Cloud Matters: Building Global, Scalable Microservices

A hands-on experience: Learn how to build a Book Management Catalog Microservice using Java, Helidon, Jakarta NoSQL, and Oracle Cloud.

February 17, 2025

by Otavio Santana

CORE

· 3,975 Views · 5 Likes

Redis as a Primary Database for Complex Applications

This article aims to answer one question: How can Redis be used as a primary database for complex applications that need to store data in multiple formats?

February 14, 2025

by Mohammed Talib

· 7,251 Views · 4 Likes

Optimizing Database Performance in Middleware Applications

Optimize database interactions in middleware with caching, parallelism, write batching, and connection pooling to boost performance and scalability effectively.

February 14, 2025

by karandeep johar

· 6,959 Views · 5 Likes

ETL Generation Using GenAI

Learn about how GenAI automates ETL pipelines, generates code, adapts to schema changes, and improves data processes with speed, efficiency, and precision.

February 14, 2025

by Ramesh Daddala

· 6,554 Views · 2 Likes

Unlocking Local AI: Build RAG Apps Without Cloud or API Keys

In this tutorial, we will use Chipper, an open-source framework that simplifies building local RAG applications without cloud dependencies or API keys.

February 14, 2025

by Santhosh Vijayabaskar

CORE

· 6,389 Views · 3 Likes

Git Tags vs AWS Tags: A Tag-Tastic Showdown!

The ultimate showdown between Git Tags and AWS Tags, two contenders with nothing in common except their love for the word "tag." Let's see them go head-to-head.

February 14, 2025

by Raghava Dittakavi

CORE

· 4,784 Views · 1 Like

AI Summarization: Extractive and Abstractive Techniques

Learn how to create an AI-powered summarization tool using Hugging Face and OpenAI, combining extractive and abstractive methods for concise, accurate results.

February 13, 2025

by Vaibhavi Tiwari

· 4,314 Views · 1 Like

How to Backup SQL Server RDS to an S3 Bucket

Learn how to back up your SQL Server RDS database directly to an Amazon S3 bucket using msdb.dbo.rds_backup_database, with detailed examples.

February 13, 2025

by arvind toorpu

CORE

· 5,475 Views · 3 Likes

Idempotency in Distributed Systems: When and Why It Matters

This article explains idempotency in distributed systems and ensuring consistent results regardless of multiple executions, with implementation and challenges.

February 13, 2025

by Sandeep Kumar Gond

· 5,513 Views

Goose Migrations for Smooth Database Changes

Migrate safely: Goose automates DB changes, version control, and rollbacks. Minimal downtime — ideal for CI/CD and high load.

February 12, 2025

by Ilia Ivankin

· 10,047 Views

Loading XML into MongoDB

Learn how to export XML data to MongoDB using SmartXML ETL tools, simplifying the process and ensuring efficient data handling and storage.

February 12, 2025

by Luca Sanders

· 6,922 Views · 1 Like

The Right ETL Architecture for Multi-Source Data Integration

Dedicated ETL pipelines are easy to set up but hard to scale, while common pipelines offer efficiency at the cost of complexity. Know which one to choose.

February 12, 2025

by Murat Balkan

CORE

· 6,285 Views · 1 Like

Cloud Security Is a Data Problem

As cloud data evolves, we need to learn how data integration, AI, and machine learning help mitigate risks in complex cloud environments and prevent breaches.

February 12, 2025

by Ryan Henrich

· 4,368 Views · 1 Like

Upcoming DZone Events

This article highlights DZone’s upcoming events, which offer insights into key industry trends and opportunities for professionals to learn, discuss, and grow.

Updated February 12, 2025

by Alanna Lovejoy

· 9,458 Views · 4 Likes

Your Network, Your Rules: Take Charge With Own DNS

Set up your own DNS nameserver using the same open-source tools that power today's state-of-the-art public DNS nameservers.

February 12, 2025

by Mahesh Veerabathiran

· 12,496 Views · 2 Likes

How to Reorganize and Rebuild Indexes in MS SQL?

In this article, we'll explore how to reorganize and rebuild indexes in MS SQL Server to address fragmentation and improve database performance.

February 12, 2025

by Priyanka Chauhan

· 7,142 Views · 1 Like

A Comprehensive Guide to Building and Debugging Apache Doris

Learn how to compile and debug Apache Doris from source. This guide also covers setup, dependencies, troubleshooting, and debugging tips for developers.

February 12, 2025

by Michael Hayden

· 4,902 Views · 2 Likes

USA PATRIOT Act vs SecNumCloud: Which Model for the Future?

The USA PATRIOT Act grants U.S. data access abroad, while French SecNumCloud secures European sovereignty. The challenge: balancing security and sovereignty.

February 12, 2025

by Frederic Jacquet

CORE

· 5,123 Views · 1 Like

The Latest Data Engineering Topics