Big Data Resources

AsyncAPI isn't as widely adopted as OpenAPI Spec, however, it's getting significant attention in the world of everything async and distributed.

September 8, 2025

by Shashi Kumar

· 2,332 Views · 2 Likes

The Role of Data Governance in Data Strategy: Part 4

Explore the critical role of data retention in governance: reduce costs, mitigate legal and cybersecurity risks, and ensure compliance with clear policies.

September 5, 2025

by Satish Gaddipati

· 2,242 Views · 2 Likes

Observability for the Invisible: Tracing Message Drops in Kafka Pipelines

Kafka lag lies. Use Fluent Bit, OpenTelemetry, DLQs, and trace IDs to expose missing messages and harden observability in event-driven pipelines.

September 3, 2025

by Prakash Wagle

· 3,108 Views · 2 Likes

Simple Efficient Spring/Kafka Datastreams

Datastreams from Souce via Kafka to Sink done simple, easy to code, efficient to run. Deployed in Kubernetes and written in Java and Kotlin.

September 3, 2025

by Sven Loesekann

· 6,546 Views · 7 Likes

Understanding Apache Spark Join Types

Three join types in Spark data frame SQL operations are crucial for the performance of big data Apache Spark applications.

September 3, 2025

by Ram Ghadiyaram

CORE

· 4,174 Views · 5 Likes

File Systems <> Database: Full Circle

The start of the computer storage era was a file-based system, which evolved into databases; However, data advancement made file systems relevant again.

September 3, 2025

by BHUSHAN FADNIS

· 1,696 Views

Keep Your Search Cluster Fit: Essential Health Checks to Keep Elasticsearch Healthy

A search cluster in top notch state requires frequent monitoring for health stats. Let's look at some health checks to always keep your ES cluster fit.

August 29, 2025

by Govind Singh Rawat

· 2,562 Views · 2 Likes

Designing Scalable Ingestion and Access Layers for Policy and Enforcement Data

Build a scalable, low-latency architecture for ingesting and accessing policy and enforcement data using Apache Spark and in-memory data grids.

August 28, 2025

by Pankaj Taneja

· 1,442 Views · 1 Like

How Healthy Is Your Data in the Age of AI? An In-Depth Checklist to Assess Data Accuracy, Governance, and AI Readiness

This guide provides a complete checklist to assess, monitor, and improve data quality for AI success, ensuring accuracy, compliance, and long-term reliability.

August 28, 2025

by Sukanya Konatam

· 2,611 Views · 2 Likes

Implementing Scalable IoT Architectures on Azure

Microsoft’s Azure IoT platform has emerged as a leading choice, powering innovative solutions across industries — from manufacturing floors to smart buildings.

August 27, 2025

by Bhimraj Ghadge

· 3,257 Views · 4 Likes

Blockchain, AI, and Edge Computing: Redefining Modern App Development

Blockchain, AI, and edge computing are enhancing modern app development by enhancing security, performance, and automation.

August 27, 2025

by Balasubramani Murugesan

· 2,904 Views · 1 Like

Digital Twins Reborn: How AI Is Finally Fulfilling the Promise of IoT

AI is revolutionizing digital twin technology, finally overcoming IoT's integration challenges and delivering real business value across industries.

August 27, 2025

by Tom Smith

CORE

· 2,463 Views · 11 Likes

Zero-Latency Data Analytics for Modern PostgreSQL Applications

Learn in this article how to set up Amazon RDS for PostgreSQL zero-ETL integration with Amazon Redshift for near-real-time analytics using the AWS CLI.

August 26, 2025

by Harpreet Kaur Chawla

· 2,761 Views · 2 Likes

Building a 3D WebXR Game with WASI Cycles: Integrating WasmEdge, Wasmtime, and Wasmer to Invoke MongoDB, Kafka, and Oracle

Develop Fullstack WASM and WASI using all the most popular frameworks, databases, frontends, etc., from top to bottom, including source code

August 25, 2025

by Paul Parkinson

· 2,475 Views · 4 Likes

From History to the Future of AI Communication—IPC to MCP and A2A

MCP and A2A are stablishing standards while existing middleware markets are promising to provide the rails. The comparison and the outlook.

August 22, 2025

by Shashi Kumar

· 2,374 Views · 1 Like

Real-Time Model Inference With Apache Kafka and Flink for Predictive AI and GenAI

Learn how data streaming with Kafka and Flink enhances AI/ML model inference, enabling low-latency, scalable predictions in real-time business use cases.

August 22, 2025

by Kai Wähner

CORE

· 1,774 Views · 2 Likes

Data Lake, Warehouse, or Lakehouse? Rethinking the Future of Data Architecture

AI is driving data lakes, warehouses, and lakehouses to converge into open, real-time platforms powered by Iceberg, Trino, and DuckDB.

August 22, 2025

by Miguel Garcia

CORE

· 2,266 Views · 1 Like

Data Storage: The Foundation for Scalable Analytics

This article provides a blueprint to build a scalable data storage foundation using a three-step framework of 5Q, BSG, and HWC with practical application.

August 22, 2025

by Junaith Haja

· 1,356 Views · 3 Likes

Greenplum vs Apache Doris: Features, Performance, and Advantages Compared

Compare Greenplum vs. Apache Doris for MPP-based analytics. Learn which database suits real-time, high-concurrency workloads and evolving data architectures.

August 21, 2025

by li yy

· 2,542 Views · 2 Likes

Operationalizing the OWASP AI Testing Guide: Building Secure AI Foundations Through NHI Governance

Align your AI pipelines with OWASP AI Testing principles using identity-based insights to monitor, enforce, and audit secrets and token usage best practices.

August 20, 2025

by Dwayne McDaniel

· 1,418 Views · 3 Likes

The Latest Big Data Topics