Data Engineering Resources

The State of Data Streaming for Manufacturing

The state of data streaming in the manufacturing industry explores case studies from BMW, Mercedes, Michelin, and Siemens powered by Apache Kafka.

June 1, 2023

by Kai Wähner

CORE

· 2,583 Views · 2 Likes

Power of PHP Frameworks: Boosting Web Development With PHP Frameworks

PHP frameworks have revolutionized web development by equipping developers with robust tools and libraries that enhance productivity, maintainability, and scalability.

June 1, 2023

by Mahesh Sharma

· 3,705 Views · 2 Likes

Knowing and Valuing Apache Kafka’s ISR (In-Sync Replicas)

To get more clarity about ISR in Apache Kafka, we should first carefully examine the replication process in the Kafka broker.

June 1, 2023

by Gautam Goswami

CORE

· 4,566 Views · 1 Like

Chaining API Requests With API Gateway

Learn how to handle client requests that should be called in sequence with an open-source API gateway which breaks down the API workflows into more manageable steps.

Updated May 31, 2023

by Bobur Umurzokov

· 4,763 Views · 2 Likes

Never Use Credentials in a CI/CD Pipeline Again

With the proliferation of OpenID Connect, is there any reason to use stored long-term credentials? I would argue that there is not.

May 31, 2023

by Sushil Kambampati

· 3,996 Views · 1 Like

Replacing Apache Hive, Elasticsearch, and PostgreSQL With Apache Doris

How does a data service company build its data warehouse? Simplicity is the best policy. See how we increased data writing efficiency by 75%.

May 31, 2023

by Twan Wang

· 3,238 Views · 1 Like

Writing a Vector Database in a Week in Rust

Explore this article describing how a vector database was built to make use of vector embeddings for semantic indexing and entity resolution using OpenAI.

May 31, 2023

by Gavin Mendel-Gleason

· 9,863 Views · 5 Likes

Microservices With Apache Camel and Quarkus

This post proposes a microservices deployment model based on Camel, using a Java development stack, Quarkus as a runtime, and K8s as a cloud-native platform.

May 31, 2023

by Nicolas Duminil

CORE

· 14,606 Views · 8 Likes

What Is Envoy Proxy?

Envoy proxy is critical to manage and secure cloud-native and Kubernetes apps. Learn why Envoy proxy is required and its architecture, features, and benefits.

May 31, 2023

by Anas T

· 8,192 Views · 8 Likes

Building a Robust Data Engineering Pipeline in the Streaming Media Industry: An Insider’s Perspective

In this detailed and personal account, the author shared his journey of building and evolving data pipelines in the rapidly transforming streaming media industry.

May 31, 2023

by Arvind Bhardwaj

· 3,771 Views · 4 Likes

Rule-Based Prompts: How To Streamline Error Handling and Boost Team Efficiency With ChatGPT

The art of creating rule-based prompts for ChatGPT is accelerating the development of apps and services for any industry, and in this article, I share how to do it.

May 31, 2023

by Denis Avramenko

· 4,919 Views · 2 Likes

Database Integration Tests With Spring Boot and Testcontainers

In this tutorial, we'll show you how to use Testcontainers for integration testing with Spring Data JPA and a PostgreSQL database.

May 31, 2023

by Andrei Rogalenko

· 10,936 Views · 4 Likes

Constructing Real-Time Analytics: Fundamental Components and Architectural Framework — Part 1

Real-time analytics provide immediate insights on large data sets so data architects can make swift decisions and stay on the cutting edge of their industry.

May 31, 2023

by David Wang

· 3,038 Views · 1 Like

Data Exploration Using Serverless SQL Pool In Azure Synapse

In this detailed article, you will learn how to carry out data ingestion and analysis using Azure Synapse Serverless SQL Pool along with sample datasets.

May 31, 2023

by Mohan Krishna Mangamuri

· 3,407 Views · 1 Like

ChatGPT Will Not Replace Your Job, but the People Using ChatGPT as Their Assistant Will

Although generative AI is revolutionizing content creation, human creativity will remain essential for audience connection.

May 31, 2023

by Dr. Debashish Roy

· 3,120 Views · 1 Like

Deep Q-Learning Networks: Bridging the Gap from Virtual Games to Real-World Applications

A significant advancement in RL is the advent of Deep Q-Learning Networks (DQNs), which combine the power of deep learning with the strategic decision-making capabilities of Q-learning.

May 30, 2023

by Yifei Wang

CORE

· 4,929 Views · 5 Likes