Data Resources

How We Rebuilt a Legacy HBase + Elasticsearch System Using Apache Iceberg, Spark, Trino, and Doris

We replaced HBase + Elasticsearch with an Iceberg lakehouse, cutting cost and complexity while supporting analytics and near-real-time access.

March 10, 2026

by Mikhail Povolotskii

· 4,026 Views · 1 Like

Optimizing Data Loader Jobs in SQL Server: Production Implementation Strategies

Accelerate SQL Server loads with bulk ops, partitioning, columnstore, minimal logging, smart batching, and tuned server settings, reducing production load times by 3–10x.

March 10, 2026

by arvind toorpu

CORE

· 3,309 Views

AI in Patient Portals: From Digital Access to Intelligent Healthcare Experiences

Traditional patient portals are functional but limited. AI enables personalization, proactive engagement, and operational efficiency.

March 9, 2026

by Muhammed Harris Kodavath

· 3,310 Views · 1 Like

Square, SumUp, Shopify: Data Streaming for Real-Time Point-of-Sale (POS)

POS systems are transforming into real-time, AI-driven platforms, fueled by mobile payments, Kafka, and Flink to empower every retail merchant.

March 9, 2026

by Kai Wähner

CORE

· 3,372 Views

Best Practices to Make Your Data AI-Ready

In this article, I share best practices for building a high-quality data management framework to support AI initiatives.

March 9, 2026

by Mykhailo Kopyl

· 4,787 Views · 1 Like

Databricks Lakeflow Spark Declarative Pipelines Migration From Non‑Unity Catalog to Unity Catalog

Migrating DLT to Unity Catalog mainly involves updating table references, permissions, and removing path-based access while keeping pipeline logic largely unchanged.

March 4, 2026

by Seshendranath Balla Venkata

· 1,402 Views · 1 Like

Implementing Decentralized Data Architecture on Google BigQuery: From Data Mesh to AI Excellence

Traditional centralized data lakes don’t scale for AI. A Data Mesh not only decentralizes data ownership by domain but also enforces federated governance.

March 3, 2026

by Jubin Abhishek Soni

CORE

· 2,800 Views · 1 Like

How Power Automate Helps Analysts Send Alert Emails Faster and How AI Builder Takes It to the Next Level

Power Automate automates data-driven alert emails, eliminating manual dashboard checks. With AI Builder, alerts become intelligent and provides proactive decision-making.

March 3, 2026

by Harsh Patel

· 2,483 Views

5 Surprising Truths About Scaling Apache Spark

Strategies for optimizing Apache Spark performance by addressing core bottlenecks like data shuffling, join inefficiencies, and excessive data scanning.

March 3, 2026

by Anurag Malik

· 1,034 Views

Probabilistic Data Structures for Software Security

Learn how bloom filters and count-min sketch make security systems fast and scalable by trading perfect accuracy for speed and memory efficiency.

March 3, 2026

by Shashank Gollapudi

· 1,813 Views · 1 Like

Why Your "Stateless" Services Are Lying to You

“Stateless” systems aren’t. Hidden state — caches, pools, SDK retries, kernel buffers — breaks deployments and scaling. Make it explicit, externalized, and observable.

March 2, 2026

by David Iyanu Jonathan

· 1,292 Views

5 Security Considerations for Deploying AI on Edge Devices

When securing AI on edge devices, consider data protection in transit and at rest, secure OTA updates, identity and access management, and more.

March 2, 2026

by Zac Amos

· 1,468 Views

Cost Is a Distributed Systems Bug

Cloud systems scale — but unchecked, they can bankrupt you. Measure, automate, and optimize costs to keep your infrastructure resilient and your budget intact.

March 2, 2026

by David Iyanu Jonathan

· 2,174 Views

Hot Data: Where Real-Time Insight Begins

Processing hot data has significant value in the modern age, as it enables businesses to make instant decisions with low-latency, fault-tolerant, real-time systems.

February 27, 2026

by Gautam Goswami

CORE

· 2,112 Views

Backlog Black Hole: Engineering a Semantic Triage Engine at Scale

Learn to reduce duplicate bug reports with semantic search: embeddings, FAISS, and GPT-4o streamline triage, saving engineers hours on large ticket backlogs.

February 27, 2026

by Nishant Jain

· 1,170 Views

Unified Intelligence: Mastering the Azure Databricks and Azure Machine Learning Integration

Bridge the gap between Big Data and production ML. Learn to integrate Azure Databricks with Azure Machine Learning for a seamless, scalable end-to-end MLOps workflow.

February 27, 2026

by Jubin Abhishek Soni

CORE

· 1,349 Views

The Hidden Cost of Custom Logic: A Performance Showdown in Apache Spark

A deep dive into PySpark UDF performance, showing why standard Python UDFs slow pipelines and when to use Pandas UDFs or native Spark functions instead.

February 26, 2026

by Abhilash Rao Mesala

· 1,750 Views

Intelligent Load Management for LLM Calls: From Static Rate Limits to Priority-Aware "Agent QoS"

Use a fair, priority-based tool scheduler instead of static rate limits, leveraging concurrency caps, signals, abort rules, and safe degradation.

February 26, 2026

by Anusha Kovi

CORE

· 1,056 Views

Edge Computing's Infrastructure Problem: What Two Years of Factory Visits Actually Revealed

Most edge computing remains cloud-dependent, with genuine use cases limited to strict latency or connectivity needs — making it more marketing than architecture.

February 25, 2026

by Igboanugo David Ugochukwu

CORE

· 1,576 Views · 2 Likes

Building Event-Driven Data Pipelines in GCP

An analysis of modernizing event-driven data pipelines using BigQuery, Firestore, and Dataflow on Google Cloud Platform (GCP).

February 24, 2026

by Krishnam Raju Narsepalle

· 1,293 Views

The Latest Data Topics