Data Engineering Resources

After the AI Commit: What Eight Retrospectives Taught Me About Working With Coding Agents

A one-week self-study of real mobile-development sessions showed that some problems I blamed on AI were actually flaws in how I organized the collaboration.

October 29, 2026

by Anton Motorniuk

· 1,269 Views

Solving Comprehension Debt in AI-Generated Codebases

Learn how the Trace, Tag, and Time-box (TTT) framework helps teams manage comprehension debt and review AI-generated code safely.

October 29, 2026

by Pushkar Gopalakrishna

· 1,919 Views

Replay, Scheduling, and Throttling: Engineering a Self-Service Kafka Streaming Tool at Scale

A Kafka recovery script grew into self-service streaming infrastructure. The hard part: guaranteeing deterministic throughput across a distributed Spark job.

October 29, 2026

by Rajesh Gomathinayagam

· 916 Views

AI Coding Agent Guardrails: Finding Gaps in Tool Permissions

A missed file edit showed me why coding agent guardrails need tests for tool coverage, hook inputs, and the changes made on disk.

October 29, 2026

by Markus Eisele

· 1,251 Views

PII Detection Before RAG Ingestion: Screening Documents in C#

We can keep sensitive personal data out of a retrieval index by screening the exact extracted text we're about to embed and holding anything that doesn't clear the gate.

October 29, 2026

by Brian O'Neill

CORE

· 1,134 Views

Stop Writing Database Migration Scripts: Declarative Schema Sync for Java

Learn to bring declarative, desired-state schema evolution to your Java applications, eliminating migration merge conflicts and preventing accidental data loss.

October 29, 2026

by Eugene Naoumov

· 1,677 Views · 3 Likes

Why Spark Executors Leave Cores Idle on YARN: A Packing Model for Executor Sizing

Oversized executors strand CPU cores; a packing model fixes it. Count the containers that fit on a node before changing any setting.

October 28, 2026

by Vikrant Bachhav

· 1,020 Views · 4 Likes

Evals in CI: How to Catch Agent Regressions Before Production Does

Agent behavior shifts with every prompt tweak, model update, and data refresh. Run evals in CI like regression tests, or ship behavior changes blind.

October 28, 2026

by Venkat Kishore Yarram

· 1,169 Views · 1 Like

Searching Kafka Without an Index: Lessons From Building Message Search for Kafdrop

Build bounded Kafka message search into Kafdrop using Java, with timestamp-based scans, memory limits, and safe consumer configuration.

October 28, 2026

by Shreyamsh Kamate

· 1,530 Views · 1 Like

Microsoft Fabric Throttling: Diagnose Before You Scale

Learn how to diagnose Microsoft Fabric capacity throttling, identify expensive workloads, optimize CU consumption, and decide when scaling is necessary.

October 28, 2026

by Amrutha Satishkumar

· 1,087 Views · 1 Like

Designing Memory Architecture for Stateful AI Agents: Short-Term Context and Long-Term Persistence

Short-term memory answers what just happened in this conversation. Long-term memory answers what the system knows about this user.

October 27, 2026

by Kapil Shirish Bidikar

· 2,230 Views

How to Migrate From LangChainGo to Genkit Go

Every LangChainGo concept mapped to its Genkit Go equivalent, with before and after code for models, prompts, chains, tools, agents, RAG and memory.

October 27, 2026

by Xavier Portilla Edo

CORE

· 1,207 Views

MQTT, Kafka, NATS, or Zenoh? Choosing Edge-to-Cloud Messaging by Traffic Pattern

Telemetry, control, site sync, and delivery to people have different physics. Here's how MQTT, Kafka, NATS, Zenoh, event mesh, and last-mile platforms map onto them.

October 27, 2026

by Kai Wähner

CORE

· 1,381 Views

Introduction to Error Prone: How to Catch Bugs at Compile Time

Error Prone plugs into javac and fails the build on code that compiles but is almost certainly wrong. It often suggests the fix, too, and you can add your own checks.

October 27, 2026

by Hüseyin Akdoğan

CORE

· 1,385 Views

Why Vulnerability Management Fails Without Reliable Asset Data

Vulnerability management is only as good as your asset data: unknown assets go unscanned, and fewer findings can mean lost visibility.

October 27, 2026

by Alex Vakulov

CORE

· 993 Views

New York City Taxi Trips: Putting Neo4j and DuckDB to Work

We load NYC taxi trip data into Neo4j using DuckDB for aggregation, then run five queries that combine both engines to show what each does best.

October 27, 2026

by Akmal Chaudhri

CORE

· 1,077 Views · 1 Like

Giving AI Agents Safe Access to Legacy Systems: Introducing legacy2mcp

legacy2mcp is a Python toolkit that generates secure Model Context Protocol servers from legacy SOAP services with input validation and audit logging.

October 27, 2026

by Balaji Venkatasubramaniyar

CORE

· 1,005 Views

Beyond the Endpoint: How Agent Cards Enable Discovery, Interoperability, and Trust

Agent Cards enable AI agents to discover capabilities, interact securely, and collaborate with trust across multi-agent ecosystems.

October 26, 2026

by Josephine Eskaline Joyce, Ph.D

CORE

· 1,305 Views · 2 Likes

LangChainGo vs Genkit Go: Where Genkit Shines

The same production service written twice: RAG, tools, typed output, retries, fallback and tracing. 73 lines in Genkit Go against 272 in LangChainGo.

October 26, 2026

by Xavier Portilla Edo

CORE

· 811 Views

To Switch, or Not to Switch the AI Model

Switching to a cheaper AI model can increase workflow costs. Learn how to revalidate outputs, track hidden costs, and maintain quality after model changes.

October 26, 2026

by Stefan Wolpers

CORE

· 1,225 Views

The Latest Data Engineering Topics