Data Engineering Resources

A Comprehensive Guide to Protect Data, Models, and Users in the GenAI Era

This article provides practical checklists to help enterprises securely adopt GenAI.

March 27, 2025

by Boris Zaikin

CORE

· 4,065 Views

Ensuring Data Quality With Great Expectations and Databricks

Ensure data quality in pipelines with Great Expectations. Learn to integrate with Databricks, validate data, and automate checks for reliable datasets.

March 26, 2025

by Sairamakrishna BuchiReddy Karri

· 2,946 Views · 1 Like

Avoid Cross-Shard Data Movement in Distributed Databases

Learn four effective strategies to optimize distributed joins, reduce network overhead, and improve query performance in sharded databases.

March 26, 2025

by Baskar Sikkayan

· 4,459 Views · 4 Likes

Jakarta NoSQL 1.0: A Way To Bring Java and NoSQL Together

Jakarta NoSQL 1.0 standardizes NoSQL integration in Java with familiar annotations, a fluent API, and support for multiple NoSQL types—boosting productivity, portability.

March 26, 2025

by Otavio Santana

CORE

· 7,972 Views · 5 Likes

A Beginner's Guide to Spark UI: Concepts and How to Use It

Understanding the fundamentals of Spark UI is essential for optimization and debugging. Although the UI is old, the information it contains is gold.

March 26, 2025

by Saransh Arora

· 2,281 Views · 3 Likes

Lakehouse: Manus? MCP? Let’s Talk About Lakehouse and AI

This is the second article in the “Lakehouse: What’s the Big Deal?” series, where I will periodically discuss Lakehouse. Your comments and discussions are welcome.

March 26, 2025

by Mingyu Chen

· 2,603 Views · 2 Likes

How to Integrate Event-Driven Ansible With Kafka

Integrate Ansible with Kafka for real-time automation: trigger playbooks via Kafka events, enhance incident response, optimize workflows, and scale seamlessly.

March 26, 2025

by Binoj Melath Nalinakshan Nair

CORE

· 4,317 Views · 12 Likes

Building AI-Driven Intelligent Applications: A Hands-On Development Guide for Integrating GenAI Into Your Applications

This article explains how to build a GenAI-powered chatbot and provides a step-by-step guide to build an intelligent application.

March 26, 2025

by Naga Santhosh Reddy Vootukuri

CORE

· 4,231 Views · 1 Like

GenAI: From Prompt to Production

GenAI-Logic is a prompt-driven approach to describing a model and logic using natural language. Create a time tracking system and post to production in less than 2 hours.

March 25, 2025

by Tyler Band

· 5,173 Views · 7 Likes

Java Reborn: Conquering Cloud-Native and Next-Gen Concurrency

Brace yourself — Java conquers cloud-native microservices, unleashes Loom’s fast concurrency, and soars with Valhalla and Panama — future-ready and fierce!

March 25, 2025

by Alok Tibrewala

· 7,978 Views · 10 Likes

Recursive Feature Elimination in Practice

Learn about Recursive Feature Elimination (RFE) to reduce feature count, boost accuracy, prevent overfitting, and build efficient machine learning models.

March 25, 2025

by Srinivas Gurrala

· 1,688 Views

Lakehouse: Starting With Apache Doris + S3 Tables

This is the first article in the “Lakehouse: What’s the Big Deal?” series, where I will periodically discuss Lakehouse. Your comments and discussions are welcome.

March 25, 2025

by Mingyu Chen

· 2,699 Views · 3 Likes

Agentic AI and Generative AI: Revolutionizing Decision Making and Automation

This article examines both agentic and generative AI systems, their relative benefits and drawbacks, their implementation challenges with ethical risks, and more.

March 25, 2025

by Nitish Upadhyay

· 2,990 Views

Make ML Models Work: A Real-World Take on Size and Imbalance

Overcoming NLP hurdles like large models and imbalanced data, the project used TF-IDF, class weighting, and simpler models for accuracy and efficiency.

March 25, 2025

by Gowsiya Syednoor Shek

· 2,134 Views

Mastering Async Context Manager Mocking in Python Tests

Simplify testing complex asynchronous Python code by mastering async context manager mocking with Pytest, ensuring reliable and maintainable test coverage.

March 25, 2025

by Aditya Karnam Gururaj Rao

· 2,528 Views · 2 Likes

Using KRaft Kafka for Development and Kubernetes Deployment

Simplify Kafka with KRaft—ditch ZooKeeper, streamline configs for Docker and Kubernetes, and integrate easily with Spring Boot for development and deployment.

March 25, 2025

by Sven Loesekann

· 4,316 Views · 5 Likes

Video Generation With Java: Leverage Nova and Amazon Bedrock

Learn how to generate videos using Amazon Nova on Amazon Bedrock with Java. This step-by-step guide covers AWS setup and Java code for video creation.

March 25, 2025

by Danil Temnikov

CORE

· 5,979 Views · 4 Likes

How to Get Word Document Form Values Using Java

This article discusses how the DOCX file structure stores form fields and values and suggests a few Java APIs that simplify retrieving DOCX form content.

March 25, 2025

by Brian O'Neill

CORE

· 4,481 Views · 1 Like