Big Data Resources

Leveraging Apache Airflow on AWS EKS (Part 1): Foundations of Data Orchestration in the Cloud

Follow an in-depth look at Apache Airflow, and how it can address the problem of orchestrating cloud-based data.

May 8, 2024

by Karthik Rajashekaran

CORE

· 3,474 Views · 2 Likes

The Impact of Tensor Contraction on AI-Driven Healthcare Data Analysis

This advanced mathematical technique enhances the structure and accessibility of data, identical to neatly folding clothes, to maximize drawer space.

May 7, 2024

by Saigurudatta Pamulaparthyvenkata

· 1,212 Views · 5 Likes

Evaluating Message Brokers

Understanding message brokers, and comparing different message brokers like SQS, Kinesis, Kafka, and Pulsar and the use cases they support.

May 6, 2024

by rahul goel

· 4,406 Views · 6 Likes

Cross-Cluster Replication for Read-Write Separation: Story of a Grocery Store Brand

Cross-cluster replication (CCR) in Apache Doris is proven to be fast, stable, and easy to use. It secures a real-time data synchronization latency of 1 second.

May 6, 2024

by Shirley H.

· 1,639 Views · 1 Like

Dataframes With Big Data in Couchbase

In this article, we'll explore how users more comfortable with dataframes can work with their Big Data in Couchbase's Analytics Service.

May 2, 2024

by Glenn Galvizo

· 1,956 Views · 1 Like

How To Fix Kafka Consumers Due To Corrupted Log Files or Missing Log Files

This article explores recovering Kafka Brokers from corrupted data storage and recovering Kafka consumers due to missing log files.

May 2, 2024

by Ramu kakarla

· 5,693 Views · 1 Like

Data Warehouse for Data Science: Adopting Arrow Flight SQL for 10X Data Transfer

Apache Doris 2.1 supports Arrow Flight SQL protocol for reading data from Doris. It delivers tenfold speedups compared to PyMySQL and Pandas.

May 1, 2024

by Shirley H.

· 1,973 Views · 2 Likes

An Introduction to Stream Processing

As a gentle introduction to stream processing, we’ve defined what it is, how it differs from batch processing, and how computation works in stream processing.

May 1, 2024

by Federico Trotta

· 4,141 Views · 5 Likes

Uber's Success With Apache Pinot: Cost Savings and Performance Boost

Uber migrated from Elasticsearch to Apache Pinot and saw a 70% reduction in costs, an 80% reduction in CPU cores, and a decrease in page load times and query timeouts.

April 29, 2024

by Kristin Crosier

· 1,903 Views · 3 Likes

The Simplest Ingredient for Developing Real-Time Front-End Applications

How to build a real-time frontend application using Redpanda Serverless, Pusher, and Vercel. Managing streaming data, and setting up real-time communication channels.

April 29, 2024

by Christina Lin

CORE

· 4,808 Views · 4 Likes

Revisiting Observability: A Deep Dive Into the State of Monitoring, Costs, and Data Ownership in Today’s Market

Explore insights into the current challenges in monitoring, highlighting issues with instrumentation, pricing models, and data ownership.

April 26, 2024

by Mike Moore

· 2,919 Views · 3 Likes

A Complete Guide to the Real-Time Streaming Protocol (RTSP)

RTSP enables control of media streaming sessions for IoT video applications like surveillance cameras. Here's everything you need to know.

April 25, 2024

by Carsten Rhod Gregersen

· 3,050 Views · 4 Likes

Java Container Application Memory Analysis

In this article, I cover types of memory usage for a Java container application and methods to analyze memory consumption.

April 24, 2024

by Parveen Saini

· 2,937 Views · 2 Likes

Starburst Unveils Fully Managed 'Icehouse' for Near Real-Time Analytics on the Open Data Lakehouse

Icehouse on Galaxy simplifies data analytics, reduces costs, and enables AI/ML workloads with an open data lakehouse architecture.

April 24, 2024

by Tom Smith

CORE

· 2,334 Views · 2 Likes

The Importance of Understanding Time Complexity in Data Structures

Why mastering time complexity is crucial, particularly in the context of data structures, and how it influences the performance and scalability of applications.

April 23, 2024

by Michael Johnson

· 120,798 Views · 78 Likes

Storage Systems For Real-Time Personalized Recommendations

Recommendation systems personalize content with machine learning, boosting engagement. They need fast, scalable storage to handle big, changing datasets.

April 23, 2024

by Jayasekhar Konduru

· 2,523 Views · 2 Likes

Introduction to Modern AI 2024 Edition, Part 2

In this post, walk through a real-life business problem and make use of ChatGPT with Advanced Data Analysis to help with the solution.

April 19, 2024

by Joel Nylund

CORE

· 1,841 Views · 3 Likes

How to Really Build a GraphQL Server With Java

This tutorial aims to demonstrate the process of building a GraphQL server utilizing the venerable graphql-java library in Java.

April 18, 2024

by David Ventimiglia

· 4,032 Views · 5 Likes

Vector Databases: Unlocking New Dimensions in Video Streaming

Explore how vector embeddings and machine learning transform search capabilities in streaming platforms and enhance user discovery and personalization.

April 15, 2024

by Sai Mahesh vuppalapati

· 5,619 Views · 6 Likes

Real-Time Data Transfer from Apache Flink to Kafka to Druid for Analysis/Decision-Making

In this article, follow an outline of the steps to transfer processed data from Flink 1.18.1 to a Kafka 2.13-3.7.0 topic.

April 15, 2024

by Gautam Goswami

CORE

· 2,817 Views · 2 Likes

The Latest Big Data Topics