Big Data Resources

What Is Reverse ETL? Overview, Use Cases, and Key Benefits

Looking to go beyond traditional analytics? Reverse ETL is a nuanced process for businesses aiming to leverage data warehouses and other data platforms.

June 4, 2024

by Suhas Jangoan

· 6,167 Views · 2 Likes

High-Performance Storage Solutions

Pairing PostgreSQL with fast storage solutions is crucial for optimizing performance, minimizing downtime, and ensuring seamless data access.

June 3, 2024

by Sergey Platonov

· 2,580 Views · 3 Likes

An Overview of Data Pipeline Architecture

Dive into how a data pipeline helps process enormous amounts of data, key components, various architecture options, and best practices for maximum benefits.

Updated June 3, 2024

by Sreenath Devineni

CORE

· 16,683 Views · 5 Likes

Profiling Big Datasets With Apache Spark and Deequ

Struggling to maintain large dataset quality or ensure reliable data attributes in your analyses? Integrating Deequ with Spark might be the solution you need.

May 29, 2024

by Akshay Jain

· 79,696 Views · 62 Likes

Streamlining Data Warehouse Migrations From RedShift, Snowflake, and BigQuery to Apache Doris

Migrate Redshift, Snowflake, and BigQuery to Apache Doris for high-performance OLAP using tools like Apache Nifi, Talend, and Python.

May 28, 2024

by Sameer Danave

· 1,636 Views · 2 Likes

Preventing and Fixing Bad Data in Event Streams (Part 1)

In this blog, we’ll take a look at how bad data may come to be, and how we can deal with it when it comes to event streams.

May 28, 2024

by Adam Bellemare

· 4,442 Views · 3 Likes

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

Learn about ARM CPU for cost-effective and energy-efficient data streaming with Apache Kafka at the Edge and in the Cloud, including use cases.

May 27, 2024

by Kai Wähner

CORE

· 2,922 Views · 1 Like

Bridging the Gap: Exploring Innovative Paths in Data Integration

This article solely focuses on delving into understanding data integrations, associated challenges, emerging technologies, and advanced methodologies.

May 27, 2024

by Yash Mehta

· 1,334 Views · 1 Like

Apache Doris for Log and Time Series Data Analysis

NetEase has replaced Elasticsearch and InfluxDB with Apache Doris, achieving 11X query performance and saving 70% of resources.

May 25, 2024

by Shirley H.

· 2,111 Views · 1 Like

Capturing Acknowledgement in Kafka Streaming With RecordMetadata

In this article, explore RecordMetadata, and its attributes, and discover how it can be leveraged using various use cases.

May 24, 2024

by Raja Chattopadhyay

· 4,227 Views · 2 Likes

Useful Tips and Tricks for Data Scientists

In this article, I will share some of the tricks and tools that I am using to interpret the data in a fast and precise way and get useful insights from it.

May 22, 2024

by Pavel Perfilov

· 5,828 Views · 35 Likes

Cost Optimization Strategies for High-Volume Data Platforms

Managing costs in running a Big Data Platform can be very challenging. This article talks through various strategies to optimize cost at every layer of the platform.

May 22, 2024

by Mayank Singhi

· 2,381 Views · 2 Likes

A Beginner’s Guide to Snowflake Architecture

Snowflake is a cloud-based data warehousing solution that targets removing the nightmares associated with business data storage, management, and analytics.

May 22, 2024

by Saigurudatta Pamulaparthyvenkata

· 3,395 Views · 3 Likes

Using Primary, Partition, and Clustering Keys in ScyllaDB (or Cassandra)

Learn core data modeling concepts that are critical for optimizing performance in wide-column NoSQL databases.

May 20, 2024

by Guy Shtub

· 2,414 Views · 1 Like

10 Svelte Data Grids: Choose the Right One for Your Project

Explore these tips on how to choose the perfect data grid that can be seamlessly integrated into your next Svelte-based project.

May 20, 2024

by Alena Stasevich

· 6,206 Views · 3 Likes

GenAI Demo With Kafka, Flink, LangChain and OpenAI

Explore the architecture and demo for open-source data streaming using Apache Kafka and Flink with Python, LangChain, and OpenAI LLM APIs in the cloud.

May 17, 2024

by Kai Wähner

CORE

· 3,545 Views · 1 Like

Data Science: A Deep Dive Into Careers and Future Scope

Data science is a booming field with diverse career options. It involves analyzing data to uncover insights and requires both technical and soft skills.

May 16, 2024

by Manas Sadangi

· 3,013 Views · 1 Like

Comparing Pandas, Polars, and PySpark: A Benchmark Analysis

Compare different datasets of three tools - Pandas, Polars, and PySpark - with a conclusion about where the future of data processing is going.

May 16, 2024

by Nacho Corcuera

· 10,441 Views · 2 Likes

DevOps vs. DataOps vs. MLOps vs. AIOps: Comparison of All "Ops"

The acronym "Ops" has rapidly increased in IT operations in recent years. Explore different "Ops" in this explanation of DevOps, DataOps, MLOps, and AIOps.

May 16, 2024

by Ravi Kiran Mallidi

CORE

· 8,864 Views · 6 Likes

10 ChatGPT Prompts To Boost Developer Productivity

In this article, we’ll explore 10 ChatGPT prompts tailored specifically for developers and engineers to boost their productivity and streamline their workflow.

May 15, 2024

by Shahid Shaikh

· 5,792 Views · 12 Likes

The Latest Big Data Topics