Data Engineering Resources

Creating a Web Project: Key Steps to Identify Issues

Learn how to identify web project issues through performance analysis, effective debugging, and product metrics tracking.

February 17, 2025

by Filipp Shcherbanich

CORE

· 5,044 Views · 4 Likes

Have LLMs Solved the Search Problem?

LLMs, while strong in content generation, need techniques like semantic chunking and vector embeddings to address the search problem in complex data environments.

February 17, 2025

by Avi Dubey

· 1,992 Views · 1 Like

Build a Data Analytics Platform With Flask, SQL, and Redis

Building a Flask-based web app that has dynamic querying for population thresholds, Redis caching for faster queries, and secure, scalable architecture.

February 17, 2025

by Sushma Kukkadapu

· 3,231 Views · 2 Likes

From Data to Decisions: Visualizing SAP Insights With Python

SAP data insights simplified with Python: A versatile tool for advanced analytics, automation, and managing large datasets.

February 17, 2025

by Prasanna Chandran Melnatami Krishnaram

· 1,694 Views · 4 Likes

Why Cloud Matters: Building Global, Scalable Microservices

A hands-on experience: Learn how to build a Book Management Catalog Microservice using Java, Helidon, Jakarta NoSQL, and Oracle Cloud.

February 17, 2025

by Otavio Santana

CORE

· 2,984 Views · 4 Likes

Redis as a Primary Database for Complex Applications

This article aims to answer one question: How can Redis be used as a primary database for complex applications that need to store data in multiple formats?

February 14, 2025

by Mohammed Talib

· 6,234 Views · 4 Likes

Optimizing Database Performance in Middleware Applications

Optimize database interactions in middleware with caching, parallelism, write batching, and connection pooling to boost performance and scalability effectively.

February 14, 2025

by karandeep johar

· 5,807 Views · 5 Likes

ETL Generation Using GenAI

Learn about how GenAI automates ETL pipelines, generates code, adapts to schema changes, and improves data processes with speed, efficiency, and precision.

February 14, 2025

by Ramesh Daddala

· 4,370 Views · 2 Likes

Unlocking Local AI: Build RAG Apps Without Cloud or API Keys

In this tutorial, we will use Chipper, an open-source framework that simplifies building local RAG applications without cloud dependencies or API keys.

February 14, 2025

by Santhosh Vijayabaskar

CORE

· 3,826 Views · 2 Likes

Git Tags vs AWS Tags: A Tag-Tastic Showdown!

The ultimate showdown between Git Tags and AWS Tags, two contenders with nothing in common except their love for the word "tag." Let's see them go head-to-head.

February 14, 2025

by Raghava Dittakavi

CORE

· 3,866 Views · 1 Like

AI Summarization: Extractive and Abstractive Techniques

Learn how to create an AI-powered summarization tool using Hugging Face and OpenAI, combining extractive and abstractive methods for concise, accurate results.

February 13, 2025

by Vaibhavi Tiwari

· 2,688 Views · 1 Like

How to Backup SQL Server RDS to an S3 Bucket

Learn how to back up your SQL Server RDS database directly to an Amazon S3 bucket using msdb.dbo.rds_backup_database, with detailed examples.

February 13, 2025

by arvind toorpu

CORE

· 3,450 Views · 2 Likes

Idempotency in Distributed Systems: When and Why It Matters

This article explains idempotency in distributed systems and ensuring consistent results regardless of multiple executions, with implementation and challenges.

February 13, 2025

by Sandeep Kumar Gond

· 2,181 Views

Goose Migrations for Smooth Database Changes

Migrate safely: Goose automates DB changes, version control, and rollbacks. Minimal downtime — ideal for CI/CD and high load.

February 12, 2025

by Ilia Ivankin

· 3,687 Views

Loading XML into MongoDB

Learn how to export XML data to MongoDB using SmartXML ETL tools, simplifying the process and ensuring efficient data handling and storage.

February 12, 2025

by Luca Sanders

· 3,734 Views · 1 Like

The Right ETL Architecture for Multi-Source Data Integration

Dedicated ETL pipelines are easy to set up but hard to scale, while common pipelines offer efficiency at the cost of complexity. Know which one to choose.

February 12, 2025

by Murat Balkan

CORE

· 3,989 Views · 1 Like

Cloud Security Is a Data Problem

As cloud data evolves, we need to learn how data integration, AI, and machine learning help mitigate risks in complex cloud environments and prevent breaches.

February 12, 2025

by Ryan Henrich

· 3,586 Views · 1 Like

Upcoming DZone Events

This article highlights DZone’s upcoming events, which offer insights into key industry trends and opportunities for professionals to learn, discuss, and grow.

Updated February 12, 2025

by Alanna Lovejoy

· 7,857 Views · 4 Likes

Your Network, Your Rules: Take Charge With Own DNS

Set up your own DNS nameserver using the same open-source tools that power today's state-of-the-art public DNS nameservers.

February 12, 2025

by Mahesh Veerabathiran

· 10,143 Views · 2 Likes

How to Reorganize and Rebuild Indexes in MS SQL?

In this article, we'll explore how to reorganize and rebuild indexes in MS SQL Server to address fragmentation and improve database performance.

February 12, 2025

by Priyanka Chauhan

· 3,171 Views · 1 Like

The Latest Data Engineering Topics