Data Engineering Resources

Declarative Pipelines in Apache Spark 4.0

Apache Spark's declarative pipelines let you define your entire data pipeline's desired outcome, and Spark handles the execution details.

August 12, 2025

by Sandeep Bishnoi

· 7,421 Views · 7 Likes

Scoped Values: Revolutionizing Java Context Management

ScopedValue in Java offers safe, immutable context propagation with clear scoping and minimal overhead—ideal for structured concurrency and virtual threads.

August 12, 2025

by Ammar Husain

CORE

· 4,263 Views · 5 Likes

Development of System Configuration Management: Introduction

We replaced open-source SCM with our own. This series shares our experience, lessons, and advice for others facing the same choice.

August 11, 2025

by Georgii Kashintsev

· 1,988 Views · 4 Likes

Data Mesh Security: How to Protect Decentralized Data Architectures

Data mesh decentralizes data ownership, creating new security gaps. This guide covers zero trust, federated governance, and observability to secure modern architectures.

August 11, 2025

by Pranjal Sharma

· 6,158 Views · 8 Likes

DSPy Framework: A Comprehensive Technical Guide With Executable Examples

DSPy improves AI development by replacing prompt engineering with patterns. This guide explores how DSPy's features enable language model applications.

August 11, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 4,994 Views · 6 Likes

Testing Approaches for Java Enterprise Applications With Jakarta NoSQL and Jakarta Data

Learn how testing strategies like mutation and data-driven testing improve reliability and quality in Jakarta EE apps, with practical tools and examples.

August 11, 2025

by Otavio Santana

CORE

· 2,498 Views · 7 Likes

Agentic AI vs Traditional AI Agents: What Makes Them Autonomous?

Agentic AI represents a transformative leap beyond traditional AI by demonstrating autonomy, proactive goal-setting, adaptive multi-step planning, and persistent memory.

August 8, 2025

by Sairamakrishna BuchiReddy Karri

· 1,947 Views · 2 Likes

State Machines Behind the Scenes of Flight Booking and Payments

Flight bookings rely on state machines to coordinate seat holds, payments, and retries. GCP Workflows make the process reliable and resilient.

August 8, 2025

by Ravi Teja Thutari

CORE

· 2,866 Views · 3 Likes

API Testing With Playwright: A Guide for QA Engineers and Developers

Playwright enables unified API and UI testing with support for all HTTP methods, authentication, retries, and mocks—helping build reliable backend tests fast.

August 8, 2025

by Rama Mallika Kadali

· 2,731 Views · 2 Likes

Boosting Algorithms Demystified: A Deep Dive into XGBoost With Code and Explanation

XGBoost is a fast, regularized, gradient boosting framework delivering top-tier accuracy, efficiency, and interpretability on structured data tasks

August 8, 2025

by Priyam Ganguly

· 2,080 Views · 1 Like

Designing Secure APIs: A Developer’s Guide to Authentication, Rate Limiting, and Data Validation

Secure APIs with strong authentication, rate limiting, input validation, and versioning to prevent attacks, manage traffic, and ensure long-term stability.

August 8, 2025

by Surendra Pandey

· 1,867 Views · 3 Likes

Databricks DBRX vs OpenAI GPT-4o vs Claude 3: Which LLM Is Best for Enterprise Use Cases?

Compare DBRX, GPT-4o, and Claude 3 in enterprise AI. Choose between open-source control, multimodal capability, or context-aware ethical alignment.

August 8, 2025

by Sairamakrishna BuchiReddy Karri

· 2,106 Views · 1 Like

Exploring C++23: Multidimensional Subscript Operator

C++23's multidimensional subscript operator simplifies array access, improving code readability and generic programming.

August 7, 2025

by Vishal Jain

· 1,985 Views · 1 Like

JWT Policy Enforcement, Rate Limiting, IP White Listing: Using Mulesoft, API Security, Cloudhub 2.0

Learn how to enforce JWT policies in API Manager with a RAML project. Secure endpoints using Auth0, plus apply IP whitelisting and rate limiting in Anypoint.

August 7, 2025

by ARINDAM GOSWAMI

· 2,204 Views

Boost Your Coding Agent and Understand Its Reasoning with 3 Simple Prompts

Use the custom prompts from this article and the linked repository to have the agent (1) plan, (2) implement, and (3) review any code for better results.

August 7, 2025

by Mező István

· 1,540 Views · 3 Likes

Impact of Artificial Intelligence on Network Routers and Switches

AI is transforming network routers and switches by driving intelligent automation, boosting performance and security, while streamlining operational complexity.

August 7, 2025

by Nikhil Kapoor

· 2,083 Views · 4 Likes

Availability to Accountability: Running AI Workloads Responsibly in the Cloud

In this article, learn the core challenges of running AI in the cloud — availability, reliability, observability, and responsibility — and how to overcome them.

August 7, 2025

by Sam Prakash Bheri

· 2,078 Views

What the AI Coding Experience Senior Software Engineers Want

Explore existing problems with AI-assisted coding. AI tools need to give developers more structure, more control, and more ways to test and trust what gets built.

August 7, 2025

by Bobur Umurzokov

· 2,190 Views · 8 Likes

Introduction to Data-Driven Testing With JUnit 5: A Guide to Efficient and Scalable Testing

Use JUnit 5’s @ParameterizedTest with @EnumSource and @MethodSource to run tests with multiple data inputs, improve test coverage, and efficiency for robust applications.

August 7, 2025

by Otavio Santana

CORE

· 3,242 Views · 6 Likes

Understanding Agentic AI: From Simple Chatbots to Autonomous Decision-Making Systems

The term "agentic AI" has become increasingly popular in tech circles, but what does it actually mean? Learn what makes agentic AI different from RAG.

August 7, 2025

by Swati Tyagi

· 2,149 Views · 2 Likes

The Latest Data Engineering Topics