Data Engineering Resources

Accelerate Your Journey to a Modern Data Platform Using Coalesce

This article will identify key challenges organizations face today in managing data platforms, and explore how advanced ETL tools can address these challenges.

September 10, 2024

by Asia Banu Shaik

· 5,823 Views · 5 Likes

Digital Garden

This article is dedicated to the concept of a digital garden — the philosophy of publicly maintaining personal notes.

September 10, 2024

by Anton Belyaev

· 5,236 Views · 2 Likes

How Can the Scalability of MySQL Be Improved for BenchmarkSQL TPC-C Testing?

Can MySQL maintain performance without collapse at tens of thousands of concurrent connections in low-conflict scenarios of BenchmarkSQL TPC-C testing?

September 10, 2024

by Bin Wang

· 3,404 Views · 2 Likes

Mastering Multi-Cloud and Edge Data Synchronization: A Retail Use Case With KubeMQ’s Java SDK

Explore how KubeMQ’s open-source Java SDK provides a powerful solution for enterprises looking to manage data across complex multi-cloud and edge environments.

September 10, 2024

by John Vester

CORE

· 48,069 Views · 3 Likes

Top Security Flaws Hiding in Your Code Right Now and How to Fix Them

This article examines the three most common injection attack types: SQL injection, Deserialization Injection, and Logging Injection — and discusses ways to prevent them.

September 10, 2024

by Jonathan Vila

· 4,916 Views · 3 Likes

Zero to AI Hero, Part 2: Understanding Plugins in Semantic Kernel, A Deep Dive With Examples

In this tutorial, explore one of the "gears" of Semantic Kernel — plugins — and discover the work they are capable of beyond chit-chatting.

September 10, 2024

by Aneesh Gopalakrishnan

· 5,226 Views · 2 Likes

Spring Boot GoT: Game of Trace!

Start with the basics of Spring Boot logs, and learn how to configure logging, test log severity, and adjust log levels using the LoggingSystem class.

September 9, 2024

by Fernando Boaglio

· 6,598 Views · 2 Likes

Model-Driven Development and Testing

Explore how open-source ApiLogicServer (ALS) and GenAI-powered development impact the deployment of microservice applications.

September 9, 2024

by Tyler Band

· 4,708 Views · 8 Likes

What Are Logging Levels?

Many things can break, but when the logging breaks, then we are completely lost. In this article, understand how to log efficiently in distributed systems.

September 9, 2024

by Adam Furmanek

CORE

· 4,446 Views · 3 Likes

Data Storage Formats for Big Data Analytics: Performance and Cost Implications of Parquet, Avro, and ORC

This article compares the performance and cost efficiency of three storage formats Parquet, Avro, and ORC on Google Cloud Platform.

September 9, 2024

by Rahul Sarabu

· 8,397 Views · 4 Likes

Mainframe Modernization: A Comprehensive Technical Blueprint

This post explores a practical hybrid approach to mainframe workload migration to the cloud using both IBM Cloud and AWS examples.

September 6, 2024

by Manjit Chakraborty

· 6,702 Views · 1 Like

Comparison of Apache Astro and Airflow

Compare Astro and Apache Airflow, explaining their architecture, features, scalability, usability, community support, and integration capabilities.

September 6, 2024

by Krishnamurty Raju Mudunuru

· 11,375 Views · 2 Likes

Principles of Modern Data Infrastructure

Explore principles of modern data infrastructure such as scalability, high availability, speed, security, maintainability, efficiency, and developer experience.

September 6, 2024

by Joe Zhou

· 5,530 Views · 1 Like

DRY Your Apache APISIX Config

DRY is an important principle in software development. This post will show you how to apply it to Apache APISIX configuration.

September 6, 2024

by Nicolas Fränkel

· 4,451 Views · 4 Likes

The AI Security Gap: Protecting Systems in the Age of Generative AI

As GenAI adoption surges, organizations face new security risks. Learn to protect your AI systems from prompt hacking and other emerging threats.

September 5, 2024

by Tom Smith

CORE

· 5,522 Views · 1 Like

Keeping Two Multi-Master Databases Aligned With a Vector Clock

In this article, learn about an experience in keeping two different databases aligned with two different technologies by using an application-level solution.

September 5, 2024

by Claudio Guidi

CORE

· 6,929 Views · 4 Likes

Setting Up a Data Warehouse for Starlight: A Comprehensive Guide

Learn architectural considerations, essential tools, and technologies, and see sample code snippets to illustrate key steps of a data warehouse setup.

September 5, 2024

by Harsh Daiya

CORE

· 4,237 Views · 1 Like

Supercharging IaC With AI for Next-Gen Infrastructure Efficiency

This article explores some important areas where AI is reshaping IaC operations and discusses what the future may hold.

September 4, 2024

by Omry Hay

· 7,250 Views · 2 Likes

My Journey With AWS CDK and Java: What You Need To Know

Learn how Java is a very viable option when working with the AWS CDK, especially for workload teams already familiar with the language and its ecosystem.

September 4, 2024

by Jeroen Reijn

CORE

· 8,574 Views · 5 Likes

How to Conduct Effective Data Security Audits for Big Data Systems

Learn key strategies for conducting thorough data security audits in big data systems to safeguard sensitive information.

September 4, 2024

by Devin Partida

· 5,553 Views · 1 Like

The Latest Data Engineering Topics