Big Data Resources

An experience software architect runs through the concepts behind Apache Spark and gives a tutorial on how to use Spark to better analyze your data sets.

April 13, 2021

by Reza Ganji

CORE

· 39,284 Views · 21 Likes

Component Testing for Event-Driven Microservice

We will see what to have in component tests for an event-driven microservice and understand what component testing is and how it's different from unit testing.

April 9, 2021

by Priyanshu Shekhar

· 6,296 Views · 4 Likes

Benefits of Data Ingestion

This article discusses that by using appropriate data ingestion tools, companies can collect, import, process data for later use or storage in a database.

April 8, 2021

by Abhishek Sharma

· 5,869 Views · 3 Likes

Quality Assurance 101 for AI and Machine Learning

QA for AI and ML is developing fast. Learn why it matters and what data quality has to do with the accuracy of your ML models and the success of AI initiatives.

April 7, 2021

by Aleksei Chumagin

· 6,012 Views · 3 Likes

AI-First Strategy: Where To Start?

Artificial intelligence can give your business a competitive advantage and unlock tremendous opportunities that were hitherto inaccessible. Here’s a 6-step plan to developing an effective AI-first strategy.

April 5, 2021

by Mir Ali

CORE

· 6,518 Views · 5 Likes

Spring Cloud Stream Binding Kafka With EmbeddedKafkaRule Using In Tests

Spring Cloud Stream is a framework built on top of Spring Boot and Spring Integration that helps in creating event-driven or message-driven microservices.

March 31, 2021

by Vitaly Ippolitov

· 9,603 Views · 3 Likes

Is Apache Kafka a Database? The 2021 Update

Kafka is a database, providing ACID guarantees. But, it works differently than other databases and doesn't replace them. It's a complementary tool.

Updated March 31, 2021

by Kai Wähner

CORE

· 64,356 Views · 28 Likes

9 Reasons Why Data Quality and Data Governance Initiatives Fail

A Gartner survey revealed that more than 90% of all data governance initiatives fail. Let’s take a look at some of the most common reasons.

March 25, 2021

by Muhammed Akheel

· 10,644 Views · 5 Likes

How To Use Business Process Management for Orchestration of Micro-Services, People, and Robots

If your DevOps team is planning process automation, here are a few key things to know about the flexibility of BPM engines and how to implement them properly.

March 24, 2021

by Miguel Valdes

· 10,755 Views · 5 Likes

How to Run an Elastic MapReduce Job Using a Custom Jar on Amazon EMR

Learn how to develop an application using Java and the MapReduce framework for Hadoop, then execute that app in Amazon's Elastic MapReduce.

March 22, 2021

by Muhammad Ali Khojaye

· 17,095 Views · 4 Likes

20 Business Intelligence Tools and Use Cases

With more data available, it’s getting more difficult to focus on the information we really need and present it in an actionable way. We list 20 tools to help you focus.

March 21, 2021

by Ekaterina Novoseltseva

CORE

· 6,146 Views · 3 Likes

Let's Think Kafka Cluster Without Zookeeper With KIP-500

Right now, Apache Kafka utilizes Apache ZooKeeper to store its metadata. Managing a ZooKeeper cluster creates an additional burden on the infrastructure and the admins.

March 19, 2021

by Siva Prasad Rao Janapati

· 6,544 Views · 3 Likes

How to Efficiently Choose the Right Database for Your Applications

Finding the right database solution for your application is not easy. Learn how to efficiently find a database for your applications.

March 19, 2021

by Guo Leitao

· 7,212 Views · 2 Likes

How Checksum Smartly Manages Data Integrity in HDFS

In two words, data integrity can be defined as an assurance of the accuracy and consistency of data throughout the entire life cycle.

March 16, 2021

by Gautam Goswami

CORE

· 5,667 Views · 2 Likes

Why Java Is so Young After 25 Years: An Architect’s Point of View

Java completes 25 years of programming life and remains closer to developer's minds; almost 69% of the worldwide developer community remains to code in Java, even now.

Updated March 16, 2021

by Dr.Magesh Kasthuri

· 16,149 Views · 16 Likes

Health Monitoring Through Data From Wearables

Wearable IoT devices for health monitoring are presented. Opportunities and challenges for developing applications for interoperable big data are also explored.

March 16, 2021

by Thiago Nascimento

CORE

· 17,582 Views · 2 Likes

Run and Scale an Apache Spark Application on Kubernetes

Learn how to set up Apache Spark on IBM Cloud Kubernetes Service by pushing the Spark container images to IBM Cloud Container Registry....

March 15, 2021

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 8,595 Views · 5 Likes

Using Machine Learning to Automate Data Cleansing

There are stages in the data cleansing process where machine learning can not only automate workflows, but influence decisions, and achieve better results.

March 8, 2021

by Zara Ziad

· 9,622 Views · 4 Likes

Apache Kafka in a Smart City Architecture

Event streaming with Apache Kafka fits into smart city architecture for data integration and correlation in real-time, at scale, between stakeholders.

March 5, 2021

by Kai Wähner

CORE

· 31,437 Views · 13 Likes

Data Integration and ETL for Dummies (Like Me)

How to understand modern data integration and ETL from a non-technical perspective.

March 4, 2021

by David Molot

· 14,786 Views · 16 Likes

The Latest Big Data Topics