Big Data Resources

Apache Doris vs Elasticsearch: An In-Depth Comparative Analysis

Apache Doris excels in complex analytics with SQL support and high performance, while Elasticsearch is ideal for full-text search and real-time retrieval.

April 23, 2025

by Michael Hayden

· 5,005 Views · 2 Likes

A Modern Stack for Building Scalable Systems

Dive into modern stack recommendations for building robust and scalable systems. This stack is agnostic and can be integrated into all programming languages.

April 23, 2025

by Bartłomiej Żyliński

CORE

· 8,962 Views · 2 Likes

Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink

Concepts and benefits of stateless and stateful stream processing with Kafka Streams and Apache Flink vs. databases and data lakes.

April 21, 2025

by Kai Wähner

CORE

· 7,167 Views · 4 Likes

Enhancing Avro With Semantic Metadata Using Logical Types

Avro logical types add semantic meaning to primitive data types, making schemas more expressive. This blog explores their benefits and common use cases.

April 16, 2025

by Gurmeet Saran

· 6,680 Views · 2 Likes

Securing Parquet Files: Vulnerabilities, Mitigations, and Validation

Parquet files are widely used in big data ecosystems, but vulnerabilities like CVE-2025-30065 highlight the importance of secure handling.

April 15, 2025

by Vamshidhar Morusu

· 6,135 Views · 1 Like

Emerging Data Architectures: The Future of Data Management

This article explores the latest advancements in data architecture, focusing on frameworks and newer paradigms such as LakeDB and zero ETL architectures.

April 15, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 8,006 Views · 10 Likes

A Deep Dive into Apache Doris Indexes

Apache Doris offers various index types, each designed for specific needs. Let’s delve into their features and discover what drives their exceptional performance.

April 14, 2025

by Darren Xu

· 6,309 Views

How Doris + Hudi Turned the Impossible Into the Everyday

Want to know how this "data giant ship" navigates the waves? Follow this article to uncover the amazing story of Doris and Hudi, the "dream team" of data.

April 14, 2025

by Zen Hua

· 5,910 Views · 4 Likes

AWS S3 Strategies for Scalable and Secure Data Lake Storage

Optimize your Amazon S3 data lake with strategic bucket configurations, data layers, encryption, and lifecycle policies for security, efficiency, and cost savings.

April 8, 2025

by Digvijay Waghela

· 8,258 Views

Optimizing Data Storage With Hybrid Partitioned Tables in Oracle 19c

Learn how to implement Hybrid Partitioned Tables (HPT) to streamline data management, boost query performance, and cut storage costs.

April 8, 2025

by arvind toorpu

CORE

· 4,214 Views · 3 Likes

Building a Cost-Effective ELK Stack for Centralized Logging

This tutorial covers the setup of a budget-friendly, secure, and scalable ELK logging platform using Infrastructure as Code (IaC) with Terraform and Ansible.

April 4, 2025

by Pradeep Kumar Goud Gaddamidi

· 9,631 Views · 5 Likes

Integrating Apache Doris and Hudi for Data Querying and Migration

Learn how to integrate Apache Doris with Apache Hudi for efficient federated querying, real-time analytics, and seamless data migration in big data environments.

April 3, 2025

by li yy

· 5,649 Views · 1 Like

Bridging OT and IT: IIoT Middleware for Edge and Cloud With Kafka and Flink

Explore how IIoT middleware and data streaming tools like Kafka and Flink bridge OT and IT, enabling real-time integration and smarter industrial operations.

April 2, 2025

by Kai Wähner

CORE

· 4,638 Views · 4 Likes

Building Scalable Data Lake Using AWS

Build a secure, scalable AWS data lake with S3, Glue, and Lake Formation, and learn key components, best practices, and analytics for optimized insights.

April 1, 2025

by Digvijay Waghela

· 4,740 Views · 3 Likes

Doris vs Elasticsearch: A Comparison and Practical Cost Case Study

Apache Doris excels in analytics, SQL support, and cost efficiency, while Elasticsearch leads in text search but has higher storage costs and complexity.

March 31, 2025

by Darren Xu

· 3,763 Views · 2 Likes

Accurate Quantitative Analysis With ChatGPT and Azure AI Hub

I asked ChatGPT the question, ‘9.9 or 9.11, which is bigger?’ ChatGPT alone answered incorrectly, but with the help of Python, it provided the correct answer: 9.9.

March 27, 2025

by Sanjeev Kumar

· 3,788 Views · 1 Like

Self-Healing Data Pipelines: The Next Big Thing in Data Engineering?

Build self-healing data pipelines on GCP using AI-powered tools like GPT-4 and Python to detect and fix errors automatically.

March 27, 2025

by Naresh Erukulla

· 9,879 Views

A Comprehensive Guide to Protect Data, Models, and Users in the GenAI Era

This article provides practical checklists to help enterprises securely adopt GenAI.

March 27, 2025

by Boris Zaikin

CORE

· 5,353 Views · 3 Likes

Ensuring Data Quality With Great Expectations and Databricks

Ensure data quality in pipelines with Great Expectations. Learn to integrate with Databricks, validate data, and automate checks for reliable datasets.

March 26, 2025

by Sairamakrishna BuchiReddy Karri

· 5,528 Views · 2 Likes

A Beginner's Guide to Spark UI: Concepts and How to Use It

Understanding the fundamentals of Spark UI is essential for optimization and debugging. Although the UI is old, the information it contains is gold.

March 26, 2025

by Saransh Arora

· 3,488 Views · 3 Likes

The Latest Big Data Topics