Big Data Resources

Monitor Apache Kafka with Telegraf and Grafana in Azure

This article talks about monitoring Apache Kafka on Azure using Telegraf and Grafana. I will guide you on installation, setup, and running monitoring solutions.

November 16, 2021

by Praveen KG

· 7,291 Views · 2 Likes

Reporting in Microservices: How To Optimize Performance

In this article, learn about a new reporting microservice developed by Freschode to overcome reporting problems they faced.

November 15, 2021

by Karina Skrinnik

· 6,356 Views · 2 Likes

What is Data Lineage and How Can It Ensure Data Quality?

By providing a clear representation of the flow of data, a data lineage system essentially allows you to have your cake and eat it too.

November 10, 2021

by Michael Bogan

CORE

· 4,562 Views · 2 Likes

How to Use SingleStore With Spark ML for Fraud Detection

In this 3-part series of articles, we’ll see how easy it is to use the SingleStore Spark Connector and the performance benefits it provides.

November 9, 2021

by Akmal Chaudhri

CORE

· 8,351 Views · 2 Likes

Lambda Architecture is one of the most useful frameworks to design big data applications and distributed data processing systems like social network platforms.

November 5, 2021

by Phani Krishna Kollapur Gandla

· 6,247 Views · 3 Likes

Kappa Architecture is Mainstream Replacing Lambda

This blog post explores why a single real-time pipeline, called Kappa architecture, is the better fit and the role of batch processing in it without requiring Lambda.

November 3, 2021

by Kai Wähner

CORE

· 8,035 Views · 8 Likes

Confluent Kafka Installation and Demo

Learn how to insert rows to SQLite-DB and show those rows on an auto-created topic on Kafka via the JDBC-source connector.

October 26, 2021

by Ersin Gulbahar

· 6,156 Views · 3 Likes

Benchmarking AWS Graviton2 and gp3 Support for Apache Kafka

Using AWS's ACCP alongside Apache Kafka on Graviton2 instances, Kafka users can get a particularly highly performant Kafka experience

October 22, 2021

by Hendra Gunadi

· 8,457 Views · 3 Likes

Distributed Tracing With Spring-cloud-Sleuth Kafka and Zipkin

In this post, you will learn to use Spring Cloud Sleuth for distributed tracing between Spring Boot microservices and Kafka with results displayed on the Zipkin server.

October 22, 2021

by Ming Jiang

· 10,069 Views · 5 Likes

Data Fabric: What Is It and Why Do You Need It?

Data fabric is a viable strategy to help overcome the barriers that previously made it hard to access data and process it in a distributed data environment.

October 21, 2021

by John Kutay

· 6,073 Views · 2 Likes

How to Accelerate Hyper-Automation With Industrial IoT

This article will walk you through specific areas where industrial IoT integration can help accelerate your pace of hyper-automation.

October 21, 2021

by Edward Ossawa

· 16,415 Views · 3 Likes

Data Fabric vs. Data Lake: Operational Comparison

In this article, we will focus on which is the most appropriate big data store for high-scale, real-time, operational use cases – data fabric vs data lake.

October 21, 2021

by Ian Tick

· 9,912 Views · 3 Likes

How To Get Closer to Consistency in Microservice Architecture

Some scenarios in microservice-based architecture involve multiple microservices. Read how completion of the scenario depends on the task completion of each microservice.

October 20, 2021

by Reza Ganji

CORE

· 10,415 Views · 10 Likes

Data Versioning 101

Data versioning makes it easy for data consumers to repeat a process or research, compare outputs as a result of using different data versions.

October 20, 2021

by Ogundiran Ayobami

· 8,221 Views · 4 Likes

Building a Real-Time Data Warehouse With TiDB and Pravega

After researching the databases and storage systems on the market, we found a more efficient and accurate real-time data warehouse solution: Pravega + TiDB.

October 20, 2021

by Tianyi Wang

· 8,786 Views · 4 Likes

Run Hundreds of Experiments with OpenCV and Hydra

In this tutorial, we'll cover how to run multiple OpenCV feature matching tasks using Hydra

October 17, 2021

by Emil Bogomolov

· 11,219 Views · 4 Likes

Cloud Data Warehouse Comparison: Redshift vs. BigQuery vs. Azure vs. Snowflake for Real-Time Workloads

We cover the pros and cons of each of these options and dive into the factors you’ll need to consider when choosing a cloud data warehouse.

Updated October 16, 2021

by Mariana Park

· 20,798 Views · 11 Likes

What Is Categorical Data and How To Identify Them

In data science, categorical data can be considered the most usable data type. In this article, we’ll explore categorical data, types, and how to identify them.

October 13, 2021

by Billy Peterson

· 4,625 Views · 2 Likes

Essential Engineering Skills For Every Software Architect

Learn eight categories of engineering craft classifications that will help you grow as a software architect to develop depth in selected areas and awareness of others.

October 12, 2021

by Ankur Kumar

· 11,557 Views · 11 Likes

Data Governance — A QuickStart With Azure Purview

Using Data effectively is incomplete without the term data governance. Here’s every “Why? How? Where?” you need to know about Data governance and Azure Purview.

October 12, 2021

by Gaurav Agarwaal

· 5,900 Views · 2 Likes

The Latest Big Data Topics