Big Data Resources

How To Fix Kafka Consumers Due To Corrupted Log Files or Missing Log Files

This article explores recovering Kafka Brokers from corrupted data storage and recovering Kafka consumers due to missing log files.

May 2, 2024

by Ramu kakarla

· 5,489 Views · 1 Like

Data Warehouse for Data Science: Adopting Arrow Flight SQL for 10X Data Transfer

Apache Doris 2.1 supports Arrow Flight SQL protocol for reading data from Doris. It delivers tenfold speedups compared to PyMySQL and Pandas.

May 1, 2024

by Shirley H.

· 1,848 Views · 2 Likes

An Introduction to Stream Processing

As a gentle introduction to stream processing, we’ve defined what it is, how it differs from batch processing, and how computation works in stream processing.

May 1, 2024

by Federico Trotta

· 4,027 Views · 5 Likes

Uber's Success With Apache Pinot: Cost Savings and Performance Boost

Uber migrated from Elasticsearch to Apache Pinot and saw a 70% reduction in costs, an 80% reduction in CPU cores, and a decrease in page load times and query timeouts.

April 29, 2024

by Kristin Crosier

· 1,839 Views · 3 Likes

The Simplest Ingredient for Developing Real-Time Front-End Applications

How to build a real-time frontend application using Redpanda Serverless, Pusher, and Vercel. Managing streaming data, and setting up real-time communication channels.

April 29, 2024

by Christina Lin

CORE

· 4,710 Views · 4 Likes

Revisiting Observability: A Deep Dive Into the State of Monitoring, Costs, and Data Ownership in Today’s Market

Explore insights into the current challenges in monitoring, highlighting issues with instrumentation, pricing models, and data ownership.

April 26, 2024

by Mike Moore

· 2,764 Views · 3 Likes

A Complete Guide to the Real-Time Streaming Protocol (RTSP)

RTSP enables control of media streaming sessions for IoT video applications like surveillance cameras. Here's everything you need to know.

April 25, 2024

by Carsten Rhod Gregersen

· 2,962 Views · 4 Likes

Java Container Application Memory Analysis

In this article, I cover types of memory usage for a Java container application and methods to analyze memory consumption.

April 24, 2024

by Parveen Saini

· 2,778 Views · 2 Likes

Starburst Unveils Fully Managed 'Icehouse' for Near Real-Time Analytics on the Open Data Lakehouse

Icehouse on Galaxy simplifies data analytics, reduces costs, and enables AI/ML workloads with an open data lakehouse architecture.

April 24, 2024

by Tom Smith

CORE

· 2,213 Views · 2 Likes

The Importance of Understanding Time Complexity in Data Structures

Why mastering time complexity is crucial, particularly in the context of data structures, and how it influences the performance and scalability of applications.

April 23, 2024

by Michael Johnson

· 120,705 Views · 78 Likes

Storage Systems For Real-Time Personalized Recommendations

Recommendation systems personalize content with machine learning, boosting engagement. They need fast, scalable storage to handle big, changing datasets.

April 23, 2024

by Jayasekhar Konduru

· 2,372 Views · 2 Likes

Introduction to Modern AI 2024 Edition, Part 2

In this post, walk through a real-life business problem and make use of ChatGPT with Advanced Data Analysis to help with the solution.

April 19, 2024

by Joel Nylund

CORE

· 1,763 Views · 3 Likes

How to Really Build a GraphQL Server With Java

This tutorial aims to demonstrate the process of building a GraphQL server utilizing the venerable graphql-java library in Java.

April 18, 2024

by David Ventimiglia

· 3,965 Views · 5 Likes

Vector Databases: Unlocking New Dimensions in Video Streaming

Explore how vector embeddings and machine learning transform search capabilities in streaming platforms and enhance user discovery and personalization.

April 15, 2024

by Sai Mahesh vuppalapati

· 5,438 Views · 6 Likes

Real-Time Data Transfer from Apache Flink to Kafka to Druid for Analysis/Decision-Making

In this article, follow an outline of the steps to transfer processed data from Flink 1.18.1 to a Kafka 2.13-3.7.0 topic.

April 15, 2024

by Gautam Goswami

CORE

· 2,720 Views · 2 Likes

AI Frameworks for Software Engineers: TensorFlow (Part 1)

TensorFlow, a key AI framework, enables the building, training, and deploying of versatile AI models, illustrated through a simple example.

April 3, 2024

by Amardeep Singh

· 5,475 Views · 7 Likes

Ethics in the Age of AI: The Human and Moral Impact of AI

Generative AI and LLMs have ever-expanding use cases, but they also have the potential for misuse, raising fundamental questions regarding AI's human and ethical impact.

April 3, 2024

by Cate Lawrence

CORE

· 4,700 Views · 3 Likes

Optimize Data Warehouse Migration Efforts by Factor of 50

This paper outlines the challenges encountered during the migration of a large data warehouse and how these challenges were tackled through innovative solutions.

April 3, 2024

by Vaibhav Dantale

· 13,632 Views · 3 Likes