Big Data Resources

How Can Developers Drive Innovation by Combining IoT and AI?

Discover how developers can drive innovation by combining IoT and AI to create transformative solutions and unlock new opportunities across industries.

May 16, 2025

by Harris Anderson

· 4,273 Views · 2 Likes

Integrating Google BigQuery With Amazon SageMaker

This guide explains how to connect Google BigQuery to Amazon SageMaker Studio using Data Wrangler for real-time machine learning.

May 15, 2025

by Shamnad Mohamed Shaffi

· 5,592 Views · 1 Like

Optimizing Integration Workflows With Spark Structured Streaming and Cloud Services

Learn how Spark Structured Streaming and cloud services optimize real-time data integration with scalable, fault-tolerant workflows for modern applications.

May 15, 2025

by Bharath Muddarla

· 7,927 Views · 3 Likes

Customer 360: Fraud Detection in Fintech With PySpark and ML

A big data ML solution using PySpark to link bank accounts and transactions for a unified customer view (Customer 360), while detecting fraud implicitly.

May 15, 2025

by Ram Ghadiyaram

CORE

· 4,981 Views · 7 Likes

Mastering Advanced Aggregations in Spark SQL

OLAP (Online Analytical Processing) aggregation techniques in Spark SQL are used for complex, multi-level data analysis over large datasets.

May 15, 2025

by Ram Ghadiyaram

CORE

· 5,716 Views · 8 Likes

A Deep Dive Into Firmware Over the Air for IoT Devices

Dive into FOTA, its importance, and best practices to design reliable and efficient FOTA services for IoT devices.

May 13, 2025

by Shweta Agarwal

· 3,958 Views · 1 Like

Scalable, Resilient Data Orchestration: The Power of Intelligent Systems

Practical insights for software professionals seeking to design large-scale data orchestration systems that unlock actionable intelligence.

May 12, 2025

by Mahesh Veerabathiran

· 3,707 Views

How Trustworthy Is Big Data?

Big data offers potential, but trust issues arise from poor quality, security, and accuracy. Organizations need strategy, governance, and training to ensure reliability.

May 12, 2025

by Dilip Kumar Rachamalla

· 4,357 Views

Event-Driven Architectures: Designing Scalable and Resilient Cloud Solutions

Learn how to enhance scalability, resilience, and efficiency in cloud solutions using event-driven architectures with this step-by-step guide.

May 7, 2025

by Srinivas Chippagiri

CORE

· 21,000 Views · 4 Likes

Unlocking the Potential of Apache Iceberg: A Comprehensive Analysis

Organization adoption perspective and key considerations of Apache Iceberg, a high-performance open-source format for large analytic tables.

May 6, 2025

by Ram Ghadiyaram

CORE

· 5,402 Views · 6 Likes

AI, ML, and Data Science: Shaping the Future of Automation

AI, machine learning, and data science are transforming industries, driving automation, advancing innovation, and addressing challenges like data privacy and bias.

May 6, 2025

by Shafeeq Ur Rahaman

· 3,640 Views

Doris: Unifying SQL Dialects for a Seamless Data Query Ecosystem

Apache Doris breaks this barrier with its powerful SQL dialect compatibility and builds a unified data query ecosystem for users.

April 28, 2025

by Darren Xu

· 3,555 Views

Apache Doris vs Elasticsearch: An In-Depth Comparative Analysis

Apache Doris excels in complex analytics with SQL support and high performance, while Elasticsearch is ideal for full-text search and real-time retrieval.

April 23, 2025

by Michael Hayden

· 5,157 Views · 2 Likes

A Modern Stack for Building Scalable Systems

Dive into modern stack recommendations for building robust and scalable systems. This stack is agnostic and can be integrated into all programming languages.

April 23, 2025

by Bartłomiej Żyliński

CORE

· 9,034 Views · 2 Likes

Stateless vs Stateful Stream Processing With Kafka Streams and Apache Flink

Concepts and benefits of stateless and stateful stream processing with Kafka Streams and Apache Flink vs. databases and data lakes.

April 21, 2025

by Kai Wähner

CORE

· 7,278 Views · 4 Likes

Enhancing Avro With Semantic Metadata Using Logical Types

Avro logical types add semantic meaning to primitive data types, making schemas more expressive. This blog explores their benefits and common use cases.

April 16, 2025

by Gurmeet Saran

· 6,789 Views · 2 Likes

Securing Parquet Files: Vulnerabilities, Mitigations, and Validation

Parquet files are widely used in big data ecosystems, but vulnerabilities like CVE-2025-30065 highlight the importance of secure handling.

April 15, 2025

by Vamshidhar Morusu

· 6,282 Views · 1 Like

Emerging Data Architectures: The Future of Data Management

This article explores the latest advancements in data architecture, focusing on frameworks and newer paradigms such as LakeDB and zero ETL architectures.

April 15, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 8,140 Views · 10 Likes

A Deep Dive into Apache Doris Indexes

Apache Doris offers various index types, each designed for specific needs. Let’s delve into their features and discover what drives their exceptional performance.

April 14, 2025

by Darren Xu

· 6,463 Views

How Doris + Hudi Turned the Impossible Into the Everyday

Want to know how this "data giant ship" navigates the waves? Follow this article to uncover the amazing story of Doris and Hudi, the "dream team" of data.

April 14, 2025

by Zen Hua

· 5,986 Views · 4 Likes

The Latest Big Data Topics