Data Engineering Resources

The Future of DevOps

Learn about Infrastructure as Code (IaC) predictions for 2025, from AI-driven drift management to cost optimization, platform engineering, and multi-framework trends.

March 27, 2025

by Omry Hay

· 3,890 Views · 3 Likes

How to Port CV/ML Models to NPU for Faster Face Recognition

This article explains the process of porting face recognition models to Rockchip NPU for higher performance in access control systems (ACS).

March 27, 2025

by Sergey Alabugin

· 3,676 Views · 1 Like

A Comprehensive Guide to Protect Data, Models, and Users in the GenAI Era

This article provides practical checklists to help enterprises securely adopt GenAI.

March 27, 2025

by Boris Zaikin

CORE

· 5,219 Views · 3 Likes

Ensuring Data Quality With Great Expectations and Databricks

Ensure data quality in pipelines with Great Expectations. Learn to integrate with Databricks, validate data, and automate checks for reliable datasets.

March 26, 2025

by Sairamakrishna BuchiReddy Karri

· 5,281 Views · 2 Likes

Avoid Cross-Shard Data Movement in Distributed Databases

Learn four effective strategies to optimize distributed joins, reduce network overhead, and improve query performance in sharded databases.

March 26, 2025

by Baskar Sikkayan

· 6,478 Views · 5 Likes

Jakarta NoSQL 1.0: A Way To Bring Java and NoSQL Together

Jakarta NoSQL 1.0 standardizes NoSQL integration in Java with familiar annotations, a fluent API, and support for multiple NoSQL types—boosting productivity, portability.

March 26, 2025

by Otavio Santana

CORE

· 9,238 Views · 5 Likes

A Beginner's Guide to Spark UI: Concepts and How to Use It

Understanding the fundamentals of Spark UI is essential for optimization and debugging. Although the UI is old, the information it contains is gold.

March 26, 2025

by Saransh Arora

· 3,332 Views · 3 Likes

Lakehouse: Manus? MCP? Let’s Talk About Lakehouse and AI

This is the second article in the “Lakehouse: What’s the Big Deal?” series, where I will periodically discuss Lakehouse. Your comments and discussions are welcome.

March 26, 2025

by Mingyu Chen

· 3,953 Views · 2 Likes

How to Integrate Event-Driven Ansible With Kafka

Integrate Ansible with Kafka for real-time automation: trigger playbooks via Kafka events, enhance incident response, optimize workflows, and scale seamlessly.

March 26, 2025

by Binoj Melath Nalinakshan Nair

CORE

· 5,541 Views · 12 Likes

Building AI-Driven Intelligent Applications: A Hands-On Development Guide for Integrating GenAI Into Your Applications

This article explains how to build a GenAI-powered chatbot and provides a step-by-step guide to build an intelligent application.

March 26, 2025

by Naga Santhosh Reddy Vootukuri

CORE

· 5,532 Views · 1 Like

GenAI: From Prompt to Production

GenAI-Logic is a prompt-driven approach to describing a model and logic using natural language. Create a time tracking system and post to production in less than 2 hours.

March 25, 2025

by Tyler Band

· 6,086 Views · 7 Likes

Java Reborn: Conquering Cloud-Native and Next-Gen Concurrency

Brace yourself — Java conquers cloud-native microservices, unleashes Loom’s fast concurrency, and soars with Valhalla and Panama — future-ready and fierce!

March 25, 2025

by Alok Tibrewala

· 8,864 Views · 10 Likes

Recursive Feature Elimination in Practice

Learn about Recursive Feature Elimination (RFE) to reduce feature count, boost accuracy, prevent overfitting, and build efficient machine learning models.

March 25, 2025

by Srinivas Gurrala

· 3,107 Views

Lakehouse: Starting With Apache Doris + S3 Tables

This is the first article in the “Lakehouse: What’s the Big Deal?” series, where I will periodically discuss Lakehouse. Your comments and discussions are welcome.

March 25, 2025

by Mingyu Chen

· 3,835 Views · 3 Likes

Agentic AI and Generative AI: Revolutionizing Decision Making and Automation

This article examines both agentic and generative AI systems, their relative benefits and drawbacks, their implementation challenges with ethical risks, and more.

March 25, 2025

by Nitish Upadhyay

· 4,036 Views

Make ML Models Work: A Real-World Take on Size and Imbalance

Overcoming NLP hurdles like large models and imbalanced data, the project used TF-IDF, class weighting, and simpler models for accuracy and efficiency.

March 25, 2025

by Gowsiya Syednoor Shek

· 2,636 Views · 1 Like

Mastering Async Context Manager Mocking in Python Tests

Simplify testing complex asynchronous Python code by mastering async context manager mocking with Pytest, ensuring reliable and maintainable test coverage.

March 25, 2025

by Aditya Karnam Gururaj Rao

· 6,635 Views · 2 Likes

Using KRaft Kafka for Development and Kubernetes Deployment

Simplify Kafka with KRaft—ditch ZooKeeper, streamline configs for Docker and Kubernetes, and integrate easily with Spring Boot for development and deployment.

March 25, 2025

by Sven Loesekann

· 5,928 Views · 6 Likes

Video Generation With Java: Leverage Nova and Amazon Bedrock

Learn how to generate videos using Amazon Nova on Amazon Bedrock with Java. This step-by-step guide covers AWS setup and Java code for video creation.

March 25, 2025

by Danil Temnikov

CORE

· 7,884 Views · 4 Likes

How to Get Word Document Form Values Using Java

This article discusses how the DOCX file structure stores form fields and values and suggests a few Java APIs that simplify retrieving DOCX form content.

March 25, 2025

by Brian O'Neill

CORE

· 5,421 Views · 1 Like

The Latest Data Engineering Topics