Data Engineering Resources

Smart-Doc: Generating gRPC API Documentation in Java Projects

In Java microservices, gRPC simplifies communication, but API documentation is challenging. Smart-doc efficiently handles gRPC API documentation.

July 25, 2025

by sun yu

· 3,560 Views

Test Pyramid: Test Setup Best Practices

Here, I revisit the plain old test pyramid and will speak a lot about how one can structure the tests to make them more reliable and profitable.

July 25, 2025

by Bartłomiej Żyliński

CORE

· 1,602 Views · 2 Likes

Modern Data Projects Need Agile Thinking — Not Just Tech

Agile isn’t just for software. This article demonstrates how Agile methods enable data teams to adapt quickly, deliver tangible value, and avoid common project pitfalls.

July 25, 2025

by Pabitra Saikia

CORE

· 1,893 Views · 2 Likes

Data Partitioning and Bucketing: How Modern Data Systems Organize and Optimize Your Data

This article explains how partitioning and bucketing work in the big data world and talks about best practices that you can follow.

July 24, 2025

by Rajanikantarao Vellaturi

· 4,403 Views · 6 Likes

How LangChain Is Powering Next-Gen AI Apps: A Developer’s Guide in 2025

LangChain is a pivotal framework empowering developers to efficiently integrate advanced language models into AI applications.

July 24, 2025

by Sairamakrishna BuchiReddy Karri

· 3,546 Views · 1 Like

Absolute Zero: How AI Is Learning Without Data

The Absolute Zero Reasoner diverges from traditional AI learning approaches by enabling AI to learn from scratch, without the need for pre-existing human-provided data.

July 24, 2025

by Tony Siciliani

· 2,444 Views

Concurrency in Rust: Writing Safe and Efficient Code

Rust's unique ownership model and expressive type system provide safe, efficient concurrency without data races or memory safety concerns.

July 24, 2025

by Sairamakrishna BuchiReddy Karri

· 2,913 Views · 1 Like

[Part-4] Text to Action: Wake Word Detection Speech to Calendar Event

Build a hands-free voice assistant with wake word detection that converts "Hey Calendar" commands into Google Calendar events using Web Speech API and AI.

July 24, 2025

by Vivek Vellaiyappan Surulimuthu

· 2,040 Views · 2 Likes

Deploying Containers on Azure Container Apps

In this article, we’ll explore Azure Container Apps and walk through example ARM templates to demonstrate how to deploy them within an Azure environment.

July 24, 2025

by Siri Varma Vegiraju

CORE

· 3,430 Views

The Hidden Bias in AI: How Data Shapes the Ethics of Machine Learning

AI bias stems from flawed data. It can be reduced through diverse datasets, fairness checks, transparency, and ethical guidelines to ensure AI aligns with human values.

July 24, 2025

by Arjun S.

· 2,069 Views · 1 Like

Why Data Privacy Without Context Will No Longer Work in 2026

Traditional anonymization is no longer enough. Learn why enterprises now need context-aware, entity-level privacy to stay compliant and operational.

July 24, 2025

by Mouli Srini

· 2,256 Views · 3 Likes

What Is Availability? Theory, Problems, Tools, and Best Practices

What is availability? Today, I will answer that question, dive into challenges, and share best practices related to availability.

July 24, 2025

by Bartłomiej Żyliński

CORE

· 2,640 Views · 2 Likes

Building a Modern Data Platform That Delivers Real Business Value

Data modernization transforms how organizations manage and use data by aligning cloud-native tech, governance, and culture to drive value and agility.

July 23, 2025

by Amlan Patnaik

· 2,465 Views · 2 Likes

Configuring Model Context Protocol (MCP) With Amazon Q CLI

In this post I share how MCP Servers can help provide better context with Amazon Q CLI, a next generation AI Coding Assistant

July 23, 2025

by Ricardo Sueiras

· 5,386 Views · 1 Like

[Part-3] Text to Action: Adding Voice Control to Your Smart Calendar

In Part 3 of “Text to Action,” we add voice commands to our calendar assistant, converting speech to events using the Web Speech API and NLP pipeline.

July 23, 2025

by Vivek Vellaiyappan Surulimuthu

· 2,054 Views · 1 Like

Building Resilient Event-Driven Applications Using Temporal

Temporal simplifies building resilient, event-driven applications by handling retries, failures, and state management without external tools.

July 23, 2025

by Lakshmi Narayana Rasalay

· 5,200 Views · 2 Likes

Designing Retry-Resilient Fare Pipelines With Idempotent Event Handling

Design retry-safe fare pipelines with idempotent event handling to ensure consistency across failures, retries, and duplications.

July 23, 2025

by Ravi Teja Thutari

CORE

· 3,847 Views · 2 Likes

Automating a Web Form With Playwright MCP and MySQL MCP

Learn how to automate form filling using Playwright MCP and MySQL MCP by fetching real-time data from a database with minimal scripting.

July 23, 2025

by Kailash Pathak

CORE

· 3,690 Views

Software Engineering Trade-Offs: Why We Cannot Build the Perfect

Our job is to focus on min-maxing software engineering trade-offs to achieve the best result. I am sharing thoughts and practices on how to do it.

July 23, 2025

by Bartłomiej Żyliński

CORE

· 1,796 Views · 7 Likes

Implementing Least Privilege in AWS IAM: Principles, Practices, and Automation

Applying least privilege in AWS IAM helps reduce risk, improve compliance, and secure environments by granting only necessary permissions to users and services.

July 22, 2025

by Ishwar Bansal

· 4,360 Views · 2 Likes

The Latest Data Engineering Topics