Data Resources

Integrating Apache Doris and Hudi for Data Querying and Migration

Learn how to integrate Apache Doris with Apache Hudi for efficient federated querying, real-time analytics, and seamless data migration in big data environments.

April 3, 2025

by li yy

· 2,666 Views · 1 Like

Problem Analysis in Apache Doris StreamLoad Scenarios

Troubleshoot common Apache Doris StreamLoad import errors, including missing partitions, data type mismatches, and special character issues, for smooth syncing.

April 3, 2025

by li yy

· 2,851 Views · 2 Likes

From Zero to Meme Hero: How I Built an AI-Powered Meme Generator in React

Learn how to build an AI meme generator using OpenAI, React, Fabric.js, and DALL-E 3 and create captions, design meme canvases, and optimize costs and performance.

April 3, 2025

by Mohit Menghnani

· 4,879 Views · 3 Likes

Maintaining ML Model Accuracy With Automated Drift Detection

This tutorial demonstrates how to implement automated drift detection, triggers alerts, and automatically retrains models to maintain accuracy in production environments.

April 2, 2025

by Jesse Williams

· 2,076 Views · 2 Likes

Lessons from Migrating an Oracle Database to AWS RDS

Migrating an on-prem Oracle database to AWS RDS brings cost savings and automation but comes with performance tuning limits, upgrade challenges, and DBA restrictions.

April 2, 2025

by arvind toorpu

CORE

· 3,620 Views · 2 Likes

Bridging OT and IT: IIoT Middleware for Edge and Cloud With Kafka and Flink

Explore how IIoT middleware and data streaming tools like Kafka and Flink bridge OT and IT, enabling real-time integration and smarter industrial operations.

April 2, 2025

by Kai Wähner

CORE

· 3,472 Views · 3 Likes

Building Scalable Data Lake Using AWS

Build a secure, scalable AWS data lake with S3, Glue, and Lake Formation, and learn key components, best practices, and analytics for optimized insights.

April 1, 2025

by Digvijay Waghela

· 3,213 Views · 3 Likes

Doris vs Elasticsearch: A Comparison and Practical Cost Case Study

Apache Doris excels in analytics, SQL support, and cost efficiency, while Elasticsearch leads in text search but has higher storage costs and complexity.

March 31, 2025

by Darren Xu

· 2,432 Views · 1 Like

Product Design vs Platform Design for Software Development

Compare platform and product design approaches for business needs. Understand their differences and compare them to find the best fit for your use case.

March 31, 2025

by Ravi Kiran Mallidi

CORE

· 3,064 Views · 4 Likes

Self-Healing Data Pipelines: The Next Big Thing in Data Engineering?

Build self-healing data pipelines on GCP using AI-powered tools like GPT-4 and Python to detect and fix errors automatically.

March 27, 2025

by Naresh Erukulla

· 6,804 Views

Automatic Versioning in Mobile Apps

TL;DR: This article presents an automatic versioning and migration framework for mobile app databases using schema hashes and seamless upgrades.

March 27, 2025

by Liewen Huang, Ph.D.

· 2,182 Views · 2 Likes

Optimized Metrics Generation With Metadata-Driven Dynamic SQL

Learn about a metadata-driven approach to generate metrics with dynamic SQL, enhancing scalability, flexibility, and efficiency in auditing and reporting.

March 27, 2025

by Tapan Parekh

· 2,185 Views · 1 Like

Ensuring Data Quality With Great Expectations and Databricks

Ensure data quality in pipelines with Great Expectations. Learn to integrate with Databricks, validate data, and automate checks for reliable datasets.

March 26, 2025

by Sairamakrishna BuchiReddy Karri

· 2,764 Views · 1 Like

Avoid Cross-Shard Data Movement in Distributed Databases

Learn four effective strategies to optimize distributed joins, reduce network overhead, and improve query performance in sharded databases.

March 26, 2025

by Baskar Sikkayan

· 4,300 Views · 4 Likes

Jakarta NoSQL 1.0: A Way To Bring Java and NoSQL Together

Jakarta NoSQL 1.0 standardizes NoSQL integration in Java with familiar annotations, a fluent API, and support for multiple NoSQL types—boosting productivity, portability.

March 26, 2025

by Otavio Santana

CORE

· 7,812 Views · 4 Likes

A Beginner's Guide to Spark UI: Concepts and How to Use It

Understanding the fundamentals of Spark UI is essential for optimization and debugging. Although the UI is old, the information it contains is gold.

March 26, 2025

by Saransh Arora

· 2,204 Views · 3 Likes

Recursive Feature Elimination in Practice

Learn about Recursive Feature Elimination (RFE) to reduce feature count, boost accuracy, prevent overfitting, and build efficient machine learning models.

March 25, 2025

by Srinivas Gurrala

· 1,555 Views

Lakehouse: Starting With Apache Doris + S3 Tables

This is the first article in the “Lakehouse: What’s the Big Deal?” series, where I will periodically discuss Lakehouse. Your comments and discussions are welcome.

March 25, 2025

by Mingyu Chen

· 2,191 Views · 2 Likes

Text Clustering With Deepseek Reasoning

We will develop and understand a novel approach to text clustering and explain our inference results using the DeepSeek reasoning model.

March 24, 2025

by Kalpan Dharamshi

· 2,010 Views · 2 Likes

Data Lake vs. Data Warehouse vs. Data Lakehouse

The pros and cons of the legacy data warehouse, the more recent data lake, and contemporary data lakehouse architectures.

Updated March 21, 2025

by Noa Shavit

· 6,789 Views · 5 Likes

The Latest Data Topics