Data Engineering Resources

ZapBot: Building a Chatbot With OpenAI and Gradio

Create your own AI-powered chatbot using GPT-4 and Gradio. This hands-on Python tutorial walks you through setup, code, UI, and secure API key handling.

July 16, 2025

by Vaibhavi Tiwari

· 3,061 Views · 1 Like

Compliance Automated Standard Solution (COMPASS), Part 8: Agentic AI Policy as Code for Compliance Automation With Prompt Declaration Language

Part of our compliance series—learn how Agentic AI and PDL help compliance teams turn natural language inputs into executable policy assessments at scale.

July 16, 2025

by Yuji Watanabe

· 2,224 Views · 1 Like

Decoding Database Speed: Essential Server Resources and Their Impact

Database performance hinges on CPU, memory, and storage. Monitoring and tuning these resources ensures efficient data ops like a kitchen running at peak speed.

July 15, 2025

by arvind toorpu

CORE

· 3,196 Views · 2 Likes

Dashboards Are Dead Weight Without Context: Why BI Needs More Than Visuals

BI engineers must build context-rich, code-driven dashboards that enable decisions. Actionable pipelines and semantic clarity are now the standard.

July 15, 2025

by Venkata Murali Krishna Nerusu

· 4,430 Views · 1 Like

Migrating SQL Failover Clusters Without Downtime: A Practical Guide

We migrated a live SQL failover cluster to new hardware using a rolling node replacement strategy—zero downtime, full validation, and no broken apps.

July 15, 2025

by Kishore Thota

· 3,038 Views · 2 Likes

Analysis of the Data Processing Framework of Pandas and Snowpark Pandas API

This is a process analysis of migrating existing Pandas workflows to an almost lift-and-shift approach using the Snowpark Pandas API to meet ever-growing data needs.

July 15, 2025

by Prasath Chetty Pandurangan

· 2,832 Views · 1 Like

How to Build a Real API Gateway With Spring Cloud Gateway and Eureka

Build a real-world API gateway using Spring Cloud Gateway and Eureka Server. Perfect for beginners looking to understand microservices architecture with working code.

July 15, 2025

by Vivek Rajyaguru

· 6,616 Views · 2 Likes

API Standards Are Data Standards

API standards are often created separately from an org's data standards. However, they are much more similar than many realize and, therefore, should be created together.

July 15, 2025

by Scott Sosna

CORE

· 4,239 Views · 2 Likes

Designing Configuration-Driven Apache Spark SQL ETL Jobs with Delta Lake CDC

Simplify complex ETL pipelines and enable scalable, maintainable data processing with Spark SQL and Delta Lake Change Data Capture.

July 14, 2025

by Janaki Ganapathi

· 2,156 Views

Testing Distributed Microservices Using XState

Learn how to use XState to model microservice workflows. Simplify testing, boost coverage, and debug visually using declarative state machines.

July 14, 2025

by Akash Verma

· 2,599 Views · 1 Like

Turn SQL into Conversation: Natural Language Database Queries With MCP

Exemplifies how a dedicated MCP server that is able to access databases can enable LLMs to inspect them and offer their users useful pieces of information.

July 14, 2025

by Horatiu Dan

CORE

· 3,444 Views · 8 Likes

Building Resilient Go Apps: Mocking and Testing Database Error Responses

Learn to simulate Azure Cosmos DB errors in Go to test retry logic and error handling using custom transports and policies — no real service failures needed.

July 14, 2025

by Abhishek Gupta

CORE

· 2,321 Views · 3 Likes

Understanding Time Series Databases

Time series databases (TSDBs) are specialized database systems optimized for storing, retrieving, and analyzing chronological data.

July 14, 2025

by Nikhil Makhija

· 3,026 Views · 3 Likes

Vibe Coding: Conversational Software Development — Part 2: In Practice

AI is transforming software development in Part 2 of my 'Vibe Coding' series. Explore conversational coding, code generation in practice, and developer productivity.

July 14, 2025

by Gaurav Gaur

CORE

· 2,833 Views · 4 Likes

The 7 Biggest Cloud Misconfigurations That Hackers Love (and How to Fix Them)

Everyone's an admin until they shouldn't be. Ninety-nine percent of cloud identities have excessive permissions, making lateral movement trivial for attackers.

July 14, 2025

by David Iyanu Jonathan

· 3,528 Views · 5 Likes

Cloud Hardware Diagnostics for AI Workloads

AI workloads need reliable hardware. Cloud providers are developing intelligent diagnostics to predict, detect, and resolve GPU and server failures efficiently.

July 14, 2025

by Sam Prakash Bheri

· 2,422 Views

AI-Powered Ransomware and Malware Detection in Cloud Environments

AI is reshaping malware and ransomware detection in cloud environments. It reviews core detection models, highlights technical challenges, and discusses future directions

July 14, 2025

by Eugene Aryeetey

· 1,907 Views

My Dive into Local LLMs, Part 2: Taming Personal Finance with Homegrown AI (and Why Privacy Matters)

Turn your local LLM (Llama 3 + Ollama on Ubuntu) into a private finance analyzer that categorizes spending and generates insights—no cloud, full privacy.

July 14, 2025

by Praveen Chinnusamy

· 15,702 Views · 5 Likes

How to Reduce Technical Debt With Artificial Intelligence (AI)

Technical debt hampers growth and innovation. Smart AI integration in SaaS helps reduce it by improving code quality, scalability, and proactive maintenance.

July 11, 2025

by Karan Rawal

· 3,263 Views

MongoDB Change Streams and Go

Change streams allow you to subscribe to real-time updates in your MongoDB collections and databases. Learn how to work with change streams and Go.

July 11, 2025

by Ado Kukic

· 2,604 Views · 1 Like

The Latest Data Engineering Topics