Data Engineering Resources

Batch Processing: 4 Tactics to Make It Cost-Efficient and Reliable

Use KEDA for cost-efficient batch processing in Kubernetes, run jobs on spot instances, ensure reentrancy, use thin messaging, and automate these processes.

August 10, 2023

by Leon Kuperman

· 2,646 Views · 1 Like

Parsing Complex JSON in Kafka Source Using Apache SeaTunnel

In conclusion, it is recommended to use the extension of UDF functions to parse the nested JSON data from Kafka sources.

August 10, 2023

by Debra Chen

· 2,575 Views · 1 Like

Ordering in Event-Sourced Systems

Ordering is extremely important in Event-Sourced Systems. Let's find out how what guarantees Event Store can provide in terms of ordering!

August 10, 2023

by Milan Savic

· 2,098 Views · 1 Like

Revolutionizing IT: Ultraviolet’s Innovative Approach to Developer Efficiency

How microservices, 'as code' methodologies, and developer-centric content creation are paving the way for the future of IT.

August 10, 2023

by Tom Smith

CORE

· 4,473 Views · 2 Likes

Cloud-Nomad Architecture

Learn about repatriation and consolidation trends, why it's a step forward, not a step back, and the core idea of cloud-nomad architecture.

August 10, 2023

by Steve Fenton

· 5,391 Views · 6 Likes

Demystifying Data Crawling: A Comprehensive Guide for Beginners

Unlock the basics of data crawling in our beginner's guide. Learn the process and importance of gathering valuable information effectively.

August 9, 2023

by Tanner Rorak

· 3,398 Views · 1 Like

AI Prowess: Harnessing Docker for Streamlined Deployment and Scalability of Machine Learning Applications

Leveraging Docker's power: streamlining deployment solutions, ensuring scalability, and simplifying CI/CD processes for machine learning models.

August 9, 2023

by Bidyut Sarkar

· 12,253 Views · 10 Likes

Low-Code Development

Explore low-code development's benefits, features, and real-world applications. Discover the power of this innovative approach.

August 9, 2023

by Julie Moore

· 4,784 Views · 3 Likes

Integrating AI Into the IDEs

Several ideas can be explored to achieve the seamless integration of AI into IDEs: some have already been implemented, and others remain untapped opportunities.

August 9, 2023

by Gregory Ledenev

· 6,550 Views · 4 Likes

SQL Server to Postgres Database Migration

Need help migrating your database from SQL Server to PostgreSQL? We've got you covered with useful tips for a smooth transition.

August 9, 2023

by Vlad Moroz

· 4,872 Views · 3 Likes

Messaging and Event Streaming: Similar Technologies, but With Very Different Characteristics

To transmit information asynchronously, choose between messaging and event streaming. They have similarities, but significant differences, so choose carefully.

August 8, 2023

by Kim Clark

· 4,933 Views · 4 Likes

Streamlit: Empowering Data Scientists With Interactive Web Applications

Streamlit is a powerful python library that allows data scientists to focus on their data and models without worrying about the intricacies of web development.

August 8, 2023

by Prateek Sharma

· 3,685 Views · 2 Likes

Real-Time Data Processing Pipeline With MongoDB, Kafka, Debezium and RisingWave

This article will cover the powerful combination of MongoDB, Kafka, Debezium, and RisingWave to analyze real-time data, how they work together.

August 8, 2023

by Bobur Umurzokov

· 3,669 Views · 2 Likes

How To Fix SQL Server Error 18456 in Simple Ways

This article not only delves into the root causes of SQL Server error 18456 but also provides users with comprehensive solutions.

August 8, 2023

by Siddharth Sharma

· 32,298 Views · 2 Likes

Exploring OpenTelemetry Capabilities

Explore the benefits of the OpenTelemetry protocol and gain the flexibility to configure telemetry according to your needs.

August 8, 2023

by Dmitriy Bogdan

· 8,195 Views · 11 Likes

How To Use Flyway for Database Migration in Spring Boot

In this article, take a look at how to use Flyway to manage a SQL database schema in a Spring Boot application.

August 8, 2023

by Seun Matt

CORE

· 17,847 Views · 3 Likes

Apache Kafka as Workflow and Orchestration Engine

Architectures and case studies for data streaming with the Apache Kafka ecosystem as stateful workflow orchestration and BPM engine.

August 8, 2023

by Kai Wähner

CORE

· 4,309 Views · 2 Likes

Universal Implementation of BFS, DFS, Dijkstra, and A-Star Algorithms

Well-known algorithms BFS, DFS, Dijkstra, and A-Star are essentially variations of the same algorithm. I will demonstrate that with actual implementation.

August 8, 2023

by Anton Yarkov

CORE

· 5,208 Views · 6 Likes

Effective Techniques for Debugging Network Connectivity Issues in Mobile Apps

This topic explores techniques to fix network connectivity issues in mobile apps, offering practical tips for developers to debug and enhance app performance.

August 8, 2023

by Michael Chukwube

· 2,750 Views · 2 Likes

Choosing the Right Approach to Enterprise Data Pipelining

Data management is like a compass for organizations to use information effectively and access valuable insights while maintaining data accuracy.

August 8, 2023

by Yash Mehta

· 4,140 Views · 1 Like

The Latest Data Engineering Topics