Big Data Resources

AI, ML, and Data Science: Shaping the Future of Automation

AI, machine learning, and data science are transforming industries, driving automation, advancing innovation, and addressing challenges like data privacy and bias.

May 6, 2025

by Shafeeq Ur Rahaman

· 3,571 Views

Doris: Unifying SQL Dialects for a Seamless Data Query Ecosystem

Apache Doris breaks this barrier with its powerful SQL dialect compatibility and builds a unified data query ecosystem for users.

April 28, 2025

by Darren Xu

· 3,503 Views

Apache Doris vs Elasticsearch: An In-Depth Comparative Analysis

Apache Doris excels in complex analytics with SQL support and high performance, while Elasticsearch is ideal for full-text search and real-time retrieval.

April 23, 2025

by Michael Hayden

· 5,076 Views · 2 Likes

A Modern Stack for Building Scalable Systems

Dive into modern stack recommendations for building robust and scalable systems. This stack is agnostic and can be integrated into all programming languages.

April 23, 2025

by Bartłomiej Żyliński

CORE

· 8,997 Views · 2 Likes

Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink

Concepts and benefits of stateless and stateful stream processing with Kafka Streams and Apache Flink vs. databases and data lakes.

April 21, 2025

by Kai Wähner

CORE

· 7,235 Views · 4 Likes

Enhancing Avro With Semantic Metadata Using Logical Types

Avro logical types add semantic meaning to primitive data types, making schemas more expressive. This blog explores their benefits and common use cases.

April 16, 2025

by Gurmeet Saran

· 6,737 Views · 2 Likes

Securing Parquet Files: Vulnerabilities, Mitigations, and Validation

Parquet files are widely used in big data ecosystems, but vulnerabilities like CVE-2025-30065 highlight the importance of secure handling.

April 15, 2025

by Vamshidhar Morusu

· 6,199 Views · 1 Like

Emerging Data Architectures: The Future of Data Management

This article explores the latest advancements in data architecture, focusing on frameworks and newer paradigms such as LakeDB and zero ETL architectures.

April 15, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 8,052 Views · 10 Likes

A Deep Dive into Apache Doris Indexes

Apache Doris offers various index types, each designed for specific needs. Let’s delve into their features and discover what drives their exceptional performance.

April 14, 2025

by Darren Xu

· 6,375 Views

How Doris + Hudi Turned the Impossible Into the Everyday

Want to know how this "data giant ship" navigates the waves? Follow this article to uncover the amazing story of Doris and Hudi, the "dream team" of data.

April 14, 2025

by Zen Hua

· 5,938 Views · 4 Likes

AWS S3 Strategies for Scalable and Secure Data Lake Storage

Optimize your Amazon S3 data lake with strategic bucket configurations, data layers, encryption, and lifecycle policies for security, efficiency, and cost savings.

April 8, 2025

by Digvijay Waghela

· 8,391 Views

Optimizing Data Storage With Hybrid Partitioned Tables in Oracle 19c

Learn how to implement Hybrid Partitioned Tables (HPT) to streamline data management, boost query performance, and cut storage costs.

April 8, 2025

by arvind toorpu

CORE

· 4,252 Views · 3 Likes

Building a Cost-Effective ELK Stack for Centralized Logging

This tutorial covers the setup of a budget-friendly, secure, and scalable ELK logging platform using Infrastructure as Code (IaC) with Terraform and Ansible.

April 4, 2025

by Pradeep Kumar Goud Gaddamidi

· 9,677 Views · 5 Likes

Integrating Apache Doris and Hudi for Data Querying and Migration

Learn how to integrate Apache Doris with Apache Hudi for efficient federated querying, real-time analytics, and seamless data migration in big data environments.

April 3, 2025

by li yy

· 5,721 Views · 1 Like

Bridging OT and IT: IIoT Middleware for Edge and Cloud With Kafka and Flink

Explore how IIoT middleware and data streaming tools like Kafka and Flink bridge OT and IT, enabling real-time integration and smarter industrial operations.

April 2, 2025

by Kai Wähner

CORE

· 4,685 Views · 4 Likes

Building Scalable Data Lake Using AWS

Build a secure, scalable AWS data lake with S3, Glue, and Lake Formation, and learn key components, best practices, and analytics for optimized insights.

April 1, 2025

by Digvijay Waghela

· 4,803 Views · 3 Likes

Doris vs Elasticsearch: A Comparison and Practical Cost Case Study

Apache Doris excels in analytics, SQL support, and cost efficiency, while Elasticsearch leads in text search but has higher storage costs and complexity.

March 31, 2025

by Darren Xu

· 3,816 Views · 2 Likes

Accurate Quantitative Analysis With ChatGPT and Azure AI Hub

I asked ChatGPT the question, ‘9.9 or 9.11, which is bigger?’ ChatGPT alone answered incorrectly, but with the help of Python, it provided the correct answer: 9.9.

March 27, 2025

by Sanjeev Kumar

· 3,820 Views · 1 Like