Data Engineering Resources

Beyond Netflix: Why Fintech Recommendations Need a Completely Different Playbook

Financial recommendations are high-stakes—unlike a bad movie pick, poor advice can impact long-term goals like retirement. Trust and care are essential.

August 14, 2025

by Swati Tyagi

· 1,841 Views · 4 Likes

The Myth of In-Place Patching: Unpacking Protocol Buffers, FieldMasks, and the "Last Field Wins" Conundrum

Directly patching Protobuf data is generally not feasible; "last field wins" and FieldMask optimize communication, but a full read-modify-write cycle is still required.

August 14, 2025

by Prithviraj Kumar Dasari

· 4,823 Views · 4 Likes

How to Successfully Program an AI

Discover how to program an AI with practical tips on tools, data preparation, model training, and ethical deployment. Ideal for all skill levels.

August 14, 2025

by Fredson Santana

· 2,459 Views · 2 Likes

Database Choices for Real-World Applications Cheat Sheet

The article provides a structured guide to selecting databases based on data structure, query patterns, and scalability needs.

August 13, 2025

by Amritendu De

· 3,585 Views · 5 Likes

Designing Data Pipelines for Real-World Systems: A Guide to Cleaning and Validating Messy Data

Learn how to build a data sanitization pipeline to clean, validate, and monitor unstructured data to improve downstream reliability.

August 13, 2025

by Alessa Cross

· 2,524 Views · 1 Like

I Vibe Coded a PC Builder Tool Using Grok AI: Here’s What I Learned Along the Way

New to vibe coding? In this article, I share the story of how I developed a PC Builder tool using Grok AI in two weeks, along with the lessons I learned.

August 13, 2025

by Sunday Zuoke

· 6,156 Views · 3 Likes

From Red to Resolution: How I Used AI to Diagnose and Recommend Fixes for Flaky Tests

Learn how to detect, diagnose, and fix flaky tests using AI and open-source tools and boost test reliability without changing your existing QA workflow.

August 13, 2025

by Arbaz Surti

· 3,496 Views · 4 Likes

Software Security Treat or Threat? Leveraging SBOMs to Control Your Supply Chain Chaos [Infographic]

Learn about the required and recommended SBOM components — plus key security practices — to enhance your security posture along the entire software supply chain.

August 13, 2025

by DZone Editorial

· 2,078 Views · 3 Likes

Creating Serverless Applications With AWS Lambda: A Step-by-Step Guide

Learn to build scalable, cost-efficient serverless applications with AWS Lambda through this detailed, step-by-step guide.

August 13, 2025

by Bhanuprakash Madupati

· 73,850 Views · 17 Likes

How to Know an Autonomous Driver Is Safe and Reliable?

Autonomous driving needs multi-layered evaluation with synthetic scenarios, large-scale simulation, and on-road driving, each playing a critical role.

August 13, 2025

by Chinmay Jain

· 1,695 Views · 3 Likes

Secure Private Connectivity Between VMware and Object Storage: An Enterprise Architecture Guide

This guide details how to establish secure private connectivity between VMware and Object Storage using Cloud Service Endpoints.

August 13, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 2,322 Views · 4 Likes

Building a Scalable GenAI Architecture for FinTech Workflows

This guide outlines a robust, modular GenAI architecture tailored for FinTech, enabling secure, scalable AI use across credit, fraud, KYC, and compliance domains.

August 12, 2025

by Elakkiya Daivam

· 15,622 Views · 7 Likes

Real-Time Recommendations Powered by Spanner, BigQuery, and Vector Embeddings

Through vector embeddings, this BigQuery and Spanner-powered system delivers real-time, personalized recommendations by capturing nuanced user and product similarities.

August 12, 2025

by Yogesh Tewari

· 3,389 Views · 3 Likes

Deploying Real-Time Machine Learning Models in Serverless Architectures: Balancing Latency, Cost, and Performance

Learn how to deploy real-time machine learning models in serverless environments while balancing latency, cost, and performance challenges.

August 12, 2025

by Kamalesh Jain

· 2,254 Views · 4 Likes

Declarative Pipelines in Apache Spark 4.0

Apache Spark's declarative pipelines let you define your entire data pipeline's desired outcome, and Spark handles the execution details.

August 12, 2025

by Sandeep Bishnoi

· 7,500 Views · 7 Likes

Scoped Values: Revolutionizing Java Context Management

ScopedValue in Java offers safe, immutable context propagation with clear scoping and minimal overhead—ideal for structured concurrency and virtual threads.

August 12, 2025

by Ammar Husain

CORE

· 4,343 Views · 5 Likes

Development of System Configuration Management: Introduction

We replaced open-source SCM with our own. This series shares our experience, lessons, and advice for others facing the same choice.

August 11, 2025

by Georgii Kashintsev

· 2,029 Views · 4 Likes

Data Mesh Security: How to Protect Decentralized Data Architectures

Data mesh decentralizes data ownership, creating new security gaps. This guide covers zero trust, federated governance, and observability to secure modern architectures.

August 11, 2025

by Pranjal Sharma

· 6,230 Views · 8 Likes

DSPy Framework: A Comprehensive Technical Guide With Executable Examples

DSPy improves AI development by replacing prompt engineering with patterns. This guide explores how DSPy's features enable language model applications.

August 11, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 5,110 Views · 6 Likes

Testing Approaches for Java Enterprise Applications With Jakarta NoSQL and Jakarta Data

Learn how testing strategies like mutation and data-driven testing improve reliability and quality in Jakarta EE apps, with practical tools and examples.

August 11, 2025

by Otavio Santana

CORE

· 2,538 Views · 7 Likes

The Latest Data Engineering Topics