Data Engineering Resources

Exploring Real-Time Data Ingestion Into Snowflake Using CockroachDB, Redpanda, and Kafka Connect

Explore Kafka Connect as a solution to stream changefeeds into Snowflake for greater control over how messages are delivered to Snowflake.

September 10, 2024

by Artem Ervits

CORE

· 5,139 Views · 1 Like

REST and HTTP Semantics

Discover if the benefits of using PUT and POST as described in the RFC outweigh whatever reasons for using them in a way that is not RFC-compliant.

September 10, 2024

by Greg Hall

· 5,236 Views · 3 Likes

Explainable AI (XAI): How Developers Build Trust and Transparency in AI Systems

XAI is essential for making AI systems transparent, trustworthy, and ethically compliant, with developers playing a crucial role.

September 10, 2024

by Suri Nuthalapati

CORE

· 47,312 Views · 3 Likes

Transforming Continuous Delivery With Feature Flags

Integrate feature flags into CI/CD pipelines to mitigate deployment risks, ensure safer feature rollouts, and facilitate feature testing.

September 10, 2024

by Josephine Eskaline Joyce

CORE

· 5,772 Views · 8 Likes

Accelerate Your Journey to a Modern Data Platform Using Coalesce

This article will identify key challenges organizations face today in managing data platforms, and explore how advanced ETL tools can address these challenges.

September 10, 2024

by Asia Banu Shaik

· 4,195 Views · 5 Likes

Digital Garden

This article is dedicated to the concept of a digital garden — the philosophy of publicly maintaining personal notes.

September 10, 2024

by Anton Belyaev

· 4,178 Views · 2 Likes

How Can the Scalability of MySQL Be Improved for BenchmarkSQL TPC-C Testing?

Can MySQL maintain performance without collapse at tens of thousands of concurrent connections in low-conflict scenarios of BenchmarkSQL TPC-C testing?

September 10, 2024

by Bin Wang

· 2,824 Views · 1 Like

Mastering Multi-Cloud and Edge Data Synchronization: A Retail Use Case With KubeMQ’s Java SDK

Explore how KubeMQ’s open-source Java SDK provides a powerful solution for enterprises looking to manage data across complex multi-cloud and edge environments.

September 10, 2024

by John Vester

CORE

· 34,157 Views · 2 Likes

Top Security Flaws Hiding in Your Code Right Now and How to Fix Them

This article examines the three most common injection attack types: SQL injection, Deserialization Injection, and Logging Injection — and discusses ways to prevent them.

September 10, 2024

by Jonathan Vila

· 4,251 Views · 3 Likes

Zero to AI Hero, Part 2: Understanding Plugins in Semantic Kernel, A Deep Dive With Examples

In this tutorial, explore one of the "gears" of Semantic Kernel — plugins — and discover the work they are capable of beyond chit-chatting.

September 10, 2024

by Aneesh Gopalakrishnan

· 3,107 Views · 1 Like

Spring Boot GoT: Game of Trace!

Start with the basics of Spring Boot logs, and learn how to configure logging, test log severity, and adjust log levels using the LoggingSystem class.

September 9, 2024

by Fernando Boaglio

· 5,199 Views · 2 Likes

Model-Driven Development and Testing

Explore how open-source ApiLogicServer (ALS) and GenAI-powered development impact the deployment of microservice applications.

September 9, 2024

by Tyler Band

· 3,426 Views · 7 Likes

What Are Logging Levels?

Many things can break, but when the logging breaks, then we are completely lost. In this article, understand how to log efficiently in distributed systems.

September 9, 2024

by Adam Furmanek

CORE

· 3,158 Views · 2 Likes

Data Storage Formats for Big Data Analytics: Performance and Cost Implications of Parquet, Avro, and ORC

This article compares the performance and cost efficiency of three storage formats Parquet, Avro, and ORC on Google Cloud Platform.

September 9, 2024

by Rahul Sarabu

· 4,579 Views · 4 Likes

Mainframe Modernization: A Comprehensive Technical Blueprint

This post explores a practical hybrid approach to mainframe workload migration to the cloud using both IBM Cloud and AWS examples.

September 6, 2024

by Manjit Chakraborty

· 5,097 Views · 1 Like

Comparison of Apache Astro and Airflow

Compare Astro and Apache Airflow, explaining their architecture, features, scalability, usability, community support, and integration capabilities.

September 6, 2024

by Krishnamurty Raju Mudunuru

· 9,899 Views · 1 Like

Principles of Modern Data Infrastructure

Explore principles of modern data infrastructure such as scalability, high availability, speed, security, maintainability, efficiency, and developer experience.

September 6, 2024

by Joe Zhou

· 4,733 Views · 1 Like

DRY Your Apache APISIX Config

DRY is an important principle in software development. This post will show you how to apply it to Apache APISIX configuration.

September 6, 2024

by Nicolas Fränkel

CORE

· 3,790 Views · 4 Likes

The AI Security Gap: Protecting Systems in the Age of Generative AI

As GenAI adoption surges, organizations face new security risks. Learn to protect your AI systems from prompt hacking and other emerging threats.

September 5, 2024

by Tom Smith

CORE

· 4,428 Views · 1 Like

Keeping Two Multi-Master Databases Aligned With a Vector Clock

In this article, learn about an experience in keeping two different databases aligned with two different technologies by using an application-level solution.

September 5, 2024

by Claudio Guidi

CORE

· 5,515 Views · 4 Likes

The Latest Data Engineering Topics