Big Data Resources

Teradata Performance and Skew Prevention Tips

Use unique identifiers such as Teradata Primary Index to evenly distribute data across AMPs to improve database performance by 70%.

February 4, 2025

by Sudheer Kumar Lagisetty

· 4,371 Views · 10 Likes

Data Governance Essentials: Policies and Procedures (Part 6)

Learn how data quality, policies, and procedures strengthen data governance by ensuring accuracy, compliance, and security for better decision-making.

February 4, 2025

by Sukanya Konatam

· 5,655 Views · 4 Likes

All You Need to Know About Apache Spark

Apache Spark is a fast, open-source cluster computing framework for big data, supporting ML, SQL, and streaming. It’s scalable, efficient, and widely used.

February 3, 2025

by Abhishek Trehan

· 4,266 Views · 2 Likes

How Apache Flink and Apache Paimon Influence Data Streaming

Apache Flink is a crucial component of Apache Paimon since it offers the real-time processing power that enhances Paimon's strong consistency and storage features.

January 28, 2025

by Gautam Goswami

CORE

· 5,864 Views · 4 Likes

CubeFS: High-Performance Storage for Cloud-Native Apps

CubeFS is a CNCF-graduated, cloud-native distributed file system delivering scalable performance, fault tolerance, and smooth Kubernetes integration.

January 24, 2025

by Sai Sandeep Ogety

CORE

· 7,949 Views · 8 Likes

Understanding HyperLogLog for Estimating Cardinality

Learn about the probabilistic algorithm designed to estimate the cardinality of a dataset with both high accuracy and low memory usage.

January 23, 2025

by Bhala Ranganathan

CORE

· 5,535 Views · 2 Likes

OPC-UA and MQTT: A Guide to Protocols, Python Implementations

Explore two essential IoT protocols: OPC-UA for secure and structured industrial device communication and MQTT, a lightweight, real-time protocol for telemetry.

January 21, 2025

by Nikhil Makhija

· 5,838 Views · 2 Likes

Seamless Transition from Elasticsearch to OpenSearch

A step-by-step guide for migrating from Elasticsearch to OpenSearch, ensuring compatibility, performance, and cost-efficiency.

January 20, 2025

by Nilesh Jain

· 6,929 Views · 3 Likes

Building a Reactive Event-Driven App With Dead Letter Queue

Learn how to build fault-tolerant, reactive event-driven applications using Spring WebFlux, Apache Kafka, and Dead Letter Queue to handle data loss efficiently.

January 20, 2025

by Sulakshana Singh

· 4,969 Views · 4 Likes

Dark Data: Recovering the Lost Opportunities

Dark data is the vast amounts of unstructured information collected by organizations that often go unused. It includes emails, customer interactions, sensor data, etc.

January 17, 2025

by Vijay Singh Khatri

CORE

· 3,621 Views

Efficient Long-Term Trend Analysis in Presto Using Datelists

To make long-term trend analysis easier, we can leverage datelists, where we store each metric value corresponding to a date in an array in a sequential manner.

January 15, 2025

by Ajay Krishnan Prabhakaran

· 4,206 Views

Kafka vs NATS: A Comparison for Message Processing

Kafka and NATS are both popular tools for message processing. This article provides a comparison between Kafka and NATS.

January 15, 2025

by Josson Paul Kalapparambath

· 7,854 Views · 2 Likes

Data-First IDP: Driving AI Innovation in Developer Platforms

A Data-First IDP integrates governance, traceability, and quality into workflows, transforming how data is managed, enabling scalable, AI-ready ecosystems.

January 15, 2025

by Paul Gale

· 6,589 Views · 2 Likes

Custom SCD2 Implementation Using PySpark

Learn to implement Slowly Changing Dimension Type 2 (SCD2) in a data warehouse for tracking historical data, ensuring data integrity, and enabling scalability.

January 13, 2025

by Abhishek Trehan

· 4,571 Views · 1 Like

Process Mining Key Elements

This article introduces process mining, explaining its key elements and practical applications for discovering and analyzing workflows using event data.

January 13, 2025

by Puneet Malhotra

· 4,796 Views · 5 Likes

Revolutionizing Catalog Management for Data Lakehouse With Polaris Catalog

Achieve cross-query-engine interoperability and zero-data-copy architecture with Polaris Catalog, enhance data management, and streamline workflows.

January 10, 2025

by Dipankar Saha

· 4,189 Views · 1 Like

Quantum Machine Learning for Large-Scale Data-Intensive Applications

This article examines how QML can harness the principles of quantum mechanics to achieve significant computational advantages over classical approaches.

January 9, 2025

by Manasi Sharma

· 7,887 Views · 1 Like

AI Governance: Building Ethical and Transparent Systems for the Future

This article takes a deep dive into AI governance, including insights surrounding its challenges, frameworks, standards, and more.

January 9, 2025

by Sukanya Konatam

· 8,458 Views · 3 Likes

Top 5 Key Features of Apache Iceberg for Modern Data Lakes

Gain insight into key Iceberg features such as time travel, schema evolution, partition evolution, and ACID transactions with clear SQL examples and diagrams.

January 8, 2025

by Rajanikantarao Vellaturi

· 4,904 Views · 4 Likes

Understanding WebRTC Security Architecture and IoT

Learn more about how WebRTC's triple-layer security architecture protects IoT communications and creates the building blocks of secure device interactions.

January 8, 2025

by Carsten Rhod Gregersen

· 6,996 Views · 1 Like

The Latest Big Data Topics