Big Data Resources

Apache Hudi: A Deep Dive With Python Code Examples

Explore Apache Hudi, an open-source data management framework providing efficient data ingestion and real-time analytics on large-scale datasets stored in data lakes.

July 2, 2024

by Harsh Daiya

CORE

· 4,751 Views · 2 Likes

Implementing Real-Time Credit Card Fraud Detection With Apache Flink on AWS

Real-time fraud detection systems are essential for identifying and preventing fraudulent transactions as they occur. Apache Flink is useful in this scenario.

July 1, 2024

by Harsh Daiya

CORE

· 8,116 Views · 4 Likes

Partitioning Hot and Cold Data Tier in Apache Kafka Cluster for Optimal Performance

Discover how by partitioning the hot and cold data tiers in the Apache Kafka Cluster, we can optimize storage resources based on data characteristics.

June 28, 2024

by Gautam Goswami

CORE

· 10,132 Views · 6 Likes

Node.js Walkthrough: Build a Simple Event-Driven Application With Kafka

Build a real-time event-driven app with Node.js and Kafka on Heroku. Follow this step-by-step guide to set up, deploy, and manage your application efficiently.

June 27, 2024

by Alvin Lee

CORE

· 16,547 Views · 3 Likes

Data Governance: Data Integration (Part 4)

It's important to have a strong data governance framework in place when integrating data from different source systems. Learn more!

June 27, 2024

by Sukanya Konatam

· 14,202 Views · 2 Likes

Building Kafka Producer With Spring Boot

In this article, I am going to show you how to build Message Publisher using Apache Kafka and Spring Boot. First, we will talk about what Apache Kafka is.

June 26, 2024

by Sai Krishna Reddy Chityala

· 10,701 Views · 3 Likes

Cluster Logging of Telecom 5G IOT Microservice Pods

Learn how Kibana visualizes data indexed in Elasticsearch log storage for vast 5G Telecom data volumes in the Kubernetes Cloud.

June 26, 2024

by BINU SUDHAKARAN PILLAI

· 15,996 Views · 5 Likes

Pure Storage Unveils Groundbreaking Innovations in AI, Automation, and Cyber Resilience at Accelerate 2024

Pure Storage announces next-gen storage automation, AI-powered management, and advanced cyber resilience solutions at Accelerate 2024 to make storage simpler.

June 26, 2024

by Tom Smith

CORE

· 10,005 Views · 1 Like

What Is ELT?

Learn about ELT (Extract, Load, Transform), a modern data management approach that loads raw data into cloud systems for scalable, real-time transformation and analytics.

June 25, 2024

by Suhas Jangoan

· 9,338 Views · 3 Likes

When Not To Use Apache Kafka (Lightboard Video)

When not to use Apache Kafka, do's and don't's; no matter if you use open source, Confluent, Amazon MSK, Event Hubs, Redpanda, Warpstream, et al.

June 25, 2024

by Kai Wähner

CORE

· 3,122 Views · 1 Like

Advanced SQL for Data Engineering

Advanced SQL is a powerful tool that allows you to retrieve, analyze, and manipulate large amounts of data in a structured and efficient way.

June 25, 2024

by Harsh Daiya

CORE

· 7,968 Views · 7 Likes

Optimizing InfiniBand Bandwidth Utilization for NVIDIA DGX Systems Using Software RAID Solutions

Leveraging SPDK vhost-blk with software RAID on NVIDIA DGX systems significantly improves InfiniBand bandwidth utilization and performance.

June 24, 2024

by Sergey Platonov

· 2,824 Views · 1 Like

MaxLinear Empowers High-Speed Connectivity and Data Acceleration Solutions for Next-Gen Computing

Advanced connectivity solutions and Panther III storage accelerator empower software engineers to build high-performance, data-driven computing systems.

June 22, 2024

by Tom Smith

CORE

· 4,103 Views · 1 Like

Embracing NoSQL: The Future of Data Storage and Retrieval

NoSQL databases handle unstructured data, provide scalability, and offer high availability, ideal for modern, big data, and real-time applications.

June 21, 2024

by Lalithkumar Prakashchand

· 4,731 Views · 4 Likes

IoT Needs To Get Serious About Security

Security issues in IoT have gotten worse, not better, and it's time we acknowledge that and fix it. It's long past time.

June 20, 2024

by David G. Simmons

CORE

· 8,560 Views · 4 Likes

Unleashing the Power of Cloud Storage With JuiceFS

JuiceFS, a cloud-native file system, empowers enterprises with scalable, high-performance storage for data-intensive AI and analytics workloads.

June 20, 2024

by Tom Smith

CORE

· 4,763 Views · 2 Likes

Harnessing Kafka Streams for Seamless Data Tasks

Learn about the use of Kafka for convenient data exploration and dynamic data integration with its "publish-subscribe" model, organizing data into topics.

June 18, 2024

by Ilia Ivankin

· 5,021 Views · 2 Likes

Another Lifesaver for Data Engineers: Apache Doris Job Scheduler for Task Automation

The built-in Doris Job Scheduler triggers pre-defined operations efficiently and reliably. It is useful in many cases including ETL and data lake analytics.

June 17, 2024

by Shirley H.

· 2,085 Views · 1 Like