Data Engineering Resources

JMS and Kafka Compared

JMS vs. Kafka — choose the right messaging system for your distributed architecture and make informed choices for efficient communication and data streaming.

August 10, 2023

by Helen Stewart

· 5,203 Views · 2 Likes

Delving Into the Performance Features of Logical Replication in PostgreSQL 16

Explore the performance features associated with logical replication and the steps for observing these features in real time.

August 10, 2023

by Ahsan Hadi

· 2,737 Views · 1 Like

Batch Processing: 4 Tactics to Make It Cost-Efficient and Reliable

Use KEDA for cost-efficient batch processing in Kubernetes, run jobs on spot instances, ensure reentrancy, use thin messaging, and automate these processes.

August 10, 2023

by Leon Kuperman

· 2,604 Views · 1 Like

Parsing Complex JSON in Kafka Source Using Apache SeaTunnel

In conclusion, it is recommended to use the extension of UDF functions to parse the nested JSON data from Kafka sources.

August 10, 2023

by Debra Chen

· 2,511 Views · 1 Like

Ordering in Event-Sourced Systems

Ordering is extremely important in Event-Sourced Systems. Let's find out how what guarantees Event Store can provide in terms of ordering!

August 10, 2023

by Milan Savic

· 2,054 Views · 1 Like

Revolutionizing IT: Ultraviolet’s Innovative Approach to Developer Efficiency

How microservices, 'as code' methodologies, and developer-centric content creation are paving the way for the future of IT.

August 10, 2023

by Tom Smith

CORE

· 4,421 Views · 2 Likes

Cloud-Nomad Architecture

Learn about repatriation and consolidation trends, why it's a step forward, not a step back, and the core idea of cloud-nomad architecture.

August 10, 2023

by Steve Fenton

· 5,349 Views · 6 Likes

Demystifying Data Crawling: A Comprehensive Guide for Beginners

Unlock the basics of data crawling in our beginner's guide. Learn the process and importance of gathering valuable information effectively.

August 9, 2023

by Tanner Rorak

· 3,332 Views · 1 Like

AI Prowess: Harnessing Docker for Streamlined Deployment and Scalability of Machine Learning Applications

Leveraging Docker's power: streamlining deployment solutions, ensuring scalability, and simplifying CI/CD processes for machine learning models.

August 9, 2023

by Bidyut Sarkar

· 12,112 Views · 10 Likes

Low-Code Development

Explore low-code development's benefits, features, and real-world applications. Discover the power of this innovative approach.

August 9, 2023

by Julie Moore

· 4,685 Views · 3 Likes

Integrating AI Into the IDEs

Several ideas can be explored to achieve the seamless integration of AI into IDEs: some have already been implemented, and others remain untapped opportunities.

August 9, 2023

by Gregory Ledenev

· 6,430 Views · 4 Likes

SQL Server to Postgres Database Migration

Need help migrating your database from SQL Server to PostgreSQL? We've got you covered with useful tips for a smooth transition.

August 9, 2023

by Vlad Moroz

· 4,610 Views · 3 Likes

Messaging and Event Streaming: Similar Technologies, but With Very Different Characteristics

To transmit information asynchronously, choose between messaging and event streaming. They have similarities, but significant differences, so choose carefully.

August 8, 2023

by Kim Clark

· 4,881 Views · 4 Likes

Streamlit: Empowering Data Scientists With Interactive Web Applications

Streamlit is a powerful python library that allows data scientists to focus on their data and models without worrying about the intricacies of web development.

August 8, 2023

by Prateek Sharma

· 3,556 Views · 2 Likes

Real-Time Data Processing Pipeline With MongoDB, Kafka, Debezium and RisingWave

This article will cover the powerful combination of MongoDB, Kafka, Debezium, and RisingWave to analyze real-time data, how they work together.

August 8, 2023

by Bobur Umurzokov

· 3,609 Views · 2 Likes

How To Fix SQL Server Error 18456 in Simple Ways

This article not only delves into the root causes of SQL Server error 18456 but also provides users with comprehensive solutions.

August 8, 2023

by Siddharth Sharma

· 32,123 Views · 2 Likes

Exploring OpenTelemetry Capabilities

Explore the benefits of the OpenTelemetry protocol and gain the flexibility to configure telemetry according to your needs.

August 8, 2023

by Dmitriy Bogdan

· 8,076 Views · 11 Likes

How To Use Flyway for Database Migration in Spring Boot

In this article, take a look at how to use Flyway to manage a SQL database schema in a Spring Boot application.

August 8, 2023

by Seun Matt

CORE

· 17,716 Views · 3 Likes

Apache Kafka as Workflow and Orchestration Engine

Architectures and case studies for data streaming with the Apache Kafka ecosystem as stateful workflow orchestration and BPM engine.

August 8, 2023

by Kai Wähner

CORE

· 4,245 Views · 2 Likes

Universal Implementation of BFS, DFS, Dijkstra, and A-Star Algorithms

Well-known algorithms BFS, DFS, Dijkstra, and A-Star are essentially variations of the same algorithm. I will demonstrate that with actual implementation.

August 8, 2023

by Anton Yarkov

CORE

· 5,027 Views · 6 Likes

The Latest Data Engineering Topics