Big Data Resources

Horizontally Scaling the Hive Metastore Database by Migrating From MySQL to TiDB

This post shows how TiDB helps Zhihu eliminate their database bottleneck and horizontally scale their Hive Metastore database to meet the growing business needs.

September 15, 2020

by mengyu hu

· 8,072 Views · 5 Likes

Accelerated Automatic Differentiation With JAX: How Does It Stack Up Against Autograd, TensorFlow, and PyTorch?

In this article, take a look at accelerated automatic differentiation with Jax and see how it stacks up against Autograd, TensorFlow, and PyTorch.

September 10, 2020

by Kevin Vu

· 12,320 Views · 2 Likes

How To Build a Real-Time, Event-Driven Information System

Building real-time information systems help enterprises to lead the pack and become competitive — learn how with WSO2 API Manager and Kafka

September 9, 2020

by Chanaka Fernando

· 13,110 Views · 3 Likes

Distributed Saga and Resiliency of Microservices

Eventual Consistency for Microservices using Distributed Saga.

Updated September 4, 2020

by Binildas Christudas

· 9,679 Views · 7 Likes

Reading AWS S3 File Content to Kafka Topic

How to Read S3 files to Kafka topic using CamelAWSS3SourceConnector.

September 3, 2020

by Ramu kakarla

· 12,464 Views · 3 Likes

Apache Kafka and SAP ERP Integration Options

Integration of Apache Kafka and SAP systems. Overview and Trade-offs of APIs, SDKs, Tools, Kafka Connect Connectors for ECC, S4/Hana, other SAP apps.

Updated August 31, 2020

by Kai Wähner

CORE

· 16,306 Views · 8 Likes

Apache Kafka Is Not for Event Sourcing

Apache Kafka is great for delivering messages, but if you're using it for event sourcing—think again. Learn about why Kafka isn't suitable for event sourcing.

August 28, 2020

by Jesper Hammarbäck

· 20,926 Views · 8 Likes

Tutorial: Data Ingestion From Kafka to Azure Data Explorer

Use the Azure Data Explorer sink connector to ingest data from Apache Kafka.

August 26, 2020

by Abhishek Gupta

CORE

· 7,197 Views · 6 Likes

Writing .NET Core Application’s Log Into ElasticSearch With NLog

See how we can take advantage of NLog and ElasticSearch to better track and manage your .NET application's logs.

Updated August 25, 2020

by Matt Ghafouri

· 22,338 Views · 4 Likes

Using PostgreSQL pgoutput Plugin for Change Data Capture With Debezium

In this article, see a quick walkthrough on how to use the PostgreSQL pgoutput plugin for change data capture with Debezium.

August 21, 2020

by Abhishek Gupta

CORE

· 18,755 Views · 4 Likes

Why you Should and How to Archive your Kafka Data to Amazon S3

Move data from Apache Kafka to AWS S3 with Kafka Connect in order to reduce storage cost and have a Kafka data recovery option.

August 21, 2020

by Stefan Bocutiu

· 18,099 Views · 5 Likes

Top 5 Apache Kafka Courses for Beginners and Experienced Developers

A discussion of five online courses you can use to begin your journey with Apache Kafka and what you can expect to learn from each of the courses.

Updated August 20, 2020

by Javin Paul

· 40,341 Views · 18 Likes

Top 5 JavaScript Libraries to Create an Organizational Chart

Here are five options with a range of capabilities, customizations, and price points to begin building your organizational chart.

Updated August 20, 2020

by Ivan Petrenko

· 212,114 Views · 14 Likes

Salesforce Change Data Capture Streaming Data With Kafka and Snowflake Data Warehouse

In our tutorial, we are going to demonstrate how Salesforce Orders can be sent to the event bus and pushed them into a Snowflake table in near real-time.

August 18, 2020

by Istvan Szegedi

· 10,409 Views · 2 Likes

SQL Percentile Aggregates and Rollups With PostgreSQL and t-digest

In this article, see how easy it is to aggregate and create SQL based percentile rollups with PostgreSQL and t-digest histograms.

August 13, 2020

by Matt Watson

· 9,917 Views · 1 Like

Processing Large Messages With Apache Kafka

Kafka wasn't built for large messages, but files and payloads keep getting bigger. This article covers use cases, architectures, and trade-offs with Kafka.

August 13, 2020

by Kai Wähner

CORE

· 50,093 Views · 4 Likes

Top 7 Security Measures for IoT Systems

In this article, take a look at seven crucial steps for a business to uplift IoT security for preventing a data breach.

August 12, 2020

by Mohit Shah

· 13,452 Views · 2 Likes

Low Code Serverless Integration With Kafka

In this article, we will build out a scenario of publishing messages from AWS Lambda to a Kafka topic.

Updated August 12, 2020

by Pranav K

CORE

· 6,976 Views · 4 Likes

Spring Boot and Kafka Configuration Tuning

See a setup for configuration tuning in an isolated environment and to determine the Spring Boot, Kafka configuration and best practices for moderate uses.

August 11, 2020

by Amit Rathi

· 94,384 Views · 10 Likes

Kafka on Kubernetes, the Strimzi Way! (Part 4)

In this part, we will configure persistence for our Kafka cluster

August 7, 2020

by Abhishek Gupta

CORE

· 10,467 Views · 6 Likes

The Latest Big Data Topics