Data Engineering Resources

The Rise of Diskless Kafka: Rethinking Brokers, Storage, and the Kafka Protocol

Diskless Kafka stores all event data in object storage without using brokers for scalable and cost-efficient data streaming architectures.

January 9, 2026

by Kai Wähner

CORE

· 1,967 Views · 2 Likes

Mastering Fluent Bit: Developer Guide to Telemetry Pipeline Routing (Part 12)

This intro to mastering Fluent Bit covers telemetry pipeline routing mechanisms, tag-based, conditional, and label-based, with hands-on examples for developers.

January 9, 2026

by Eric D. Schabell

CORE

· 1,693 Views · 2 Likes

How to Build and Deploy an AI Agent on Kubernetes With AWS Bedrock, FastAPI and Helm

Learn how to build, containerize, and deploy a lightweight, cloud-native AI agent on Amazon EKS using FastAPI, AWS Bedrock, Docker, and Helm.

January 9, 2026

by Chandrasekhar Rao Katru

· 3,986 Views · 6 Likes

Essential Techniques for Production Vector Search Systems Part 2 - Binary Quantization

Proven techniques for production vector search including when to use each one, how to combine them effectively, trade offs to understand before deployment.

January 9, 2026

by Pavan Vemuri

CORE

· 2,106 Views · 3 Likes

Multi-Region Apache Kafka using Synchronous Replication for Disaster Recovery With Zero Data Loss (RPO=0)

Kafka isn’t one-size-fits-all. Choose between self-managed, serverless, or BYOC deployments. New RPO=0 options now enable zero data loss for real-time applications.

January 9, 2026

by Kai Wähner

CORE

· 1,725 Views · 2 Likes

From Code to Runtime: How AI Is Bridging the SAST–DAST Gap

A practical exploration of how AI connects static code vulnerabilities with their potential runtime exploitability in modern cloud-native applications

January 9, 2026

by Atish Kumar Dash

· 1,846 Views · 1 Like

How AI Is Rewriting DevOps: Practical Patterns for Faster, Safer Releases

AI is reshaping DevOps by accelerating development, testing, and deployment while enhancing reliability through smart automation.

January 9, 2026

by Bhanu Sekhar Guttikonda

CORE

· 2,281 Views · 2 Likes

When Services Think for Themselves: Traditional Orchestration vs. Agentic AI Microservices

Why traditional orchestration falls short and how agentic AI enables predictive, autonomous microservices for modern cloud-native architectures.

January 8, 2026

by Atish Kumar Dash

· 2,660 Views · 1 Like

Essential Techniques for Production Vector Search Systems Part 1 - Hybrid Search

Proven techniques for production vector search including when to use each one, how to combine them effectively, and trade offs to understand before deployment.

January 8, 2026

by Pavan Vemuri

CORE

· 2,253 Views · 2 Likes

Telemetry-Driven AI Architecture: Closing the Loop from UX to Models

Most Android AI features fail after launch because they don’t learn from real users — this architecture logs predictions and user outcomes.

January 8, 2026

by Mohan Sankaran

· 3,028 Views · 5 Likes

Building a Containerized Quarkus API and a CI/CD Pipeline on AWS EKS/Fargate with CDK

This project demonstrates a production-ready approach to deploying a containerized Quarkus API on AWS EKS/Fargate, leveraging Infrastructure as Code through the AWS CDK.

January 8, 2026

by Nicolas Duminil

CORE

· 2,486 Views · 1 Like

Secure Log Tokenization Using Aho–Corasick and Spring

This article shows how to use the Aho–Corasick algorithm and deterministic tokenization in Spring Boot to intercept logs in real time, remove sensitive values.

January 8, 2026

by Balakumaran Sugumar

· 2,275 Views · 3 Likes

Handling Logging After Migrating UiPath to Automation Cloud

Learn about logging challenges after migrating UiPath Orchestrator to Automation Cloud and robot-level log ingestion as a scalable observability approach.

January 7, 2026

by RAVINDRA REDDY MADIREDDY

· 2,819 Views · 2 Likes

AWS Bedrock vs Azure OpenAI vs Gemini API: A Practical Comparison

Bedrock = multi-model flexibility, Azure OpenAI = Microsoft-centric enterprise, Gemini = long-context and multimodal with free tier.

January 7, 2026

by Jubin Soni, FBCS

CORE

· 5,341 Views · 1 Like

Implementing Idempotency in Distributed Spring Boot Applications Using MySQL

Implementing idempotency in distributed Spring Boot applications using MySQL row-level locking and transactions to handle concurrent requests safely.

January 7, 2026

by Balakumaran Sugumar

· 6,040 Views · 7 Likes

Automated Deployment Using a CI/CD Pipeline (Mule 4 | CloudHub 2.0)

This tutorial demonstrates automated deployment using a CI/CD pipeline with Mule 4. It shows how code pushed to a Git repository triggers deployment automatically.

January 7, 2026

by ARINDAM GOSWAMI

· 2,932 Views · 1 Like

The Hidden Security Risks in ETL/ELT Pipelines for LLM-Enabled Organizations

As LLMs enter data pipelines, ETL/ELT becomes part of the AI security boundary, where untrusted inputs can introduce upstream risks.

January 7, 2026

by Vivek Venkatesan

· 4,914 Views · 2 Likes

Why Small Language Models Are Transforming AI Adoption for Everyone

Small language models (SLMs) enable faster, efficient, and on-device AI, reducing costs while making advanced AI accessible to more users and businesses.

January 7, 2026

by Sanjaykumar Ghinaiya

· 2,336 Views · 1 Like

Orchestrating Retail-Scale Data on Google Cloud

In this post, learn how to build scalable, reliable retail data pipelines with Google Cloud: Pub/Sub, Dataflow, BigQuery, and Cloud Composer in action.

January 7, 2026

by Balakrishna Aitha

· 1,304 Views

Solving the Cold Start Problem in Edge AI: A Guide to Data-Saving Learning

Update edge AI models efficiently using Mix Up and contribution sampling to overcome domain shift with minimal data, ensuring continuous evolution without forgetting.

January 6, 2026

by Dippu Kumar Singh

· 3,701 Views

The Latest Data Engineering Topics