Data Engineering Resources

Dynamic Watermarking With IMGProxy and Apache APISIX

In this post, we implement a watermarking feature with the help of imgproxy. The more I think about it, the more I think they make a match made in Heaven.

July 17, 2024

by Nicolas Fränkel

CORE

· 5,122 Views · 1 Like

AI Agentic 101: Understanding Artificial Intelligence Agents

What is agentic AI and how do these autonomous systems improve productivity, facilitate smarter decisions, and at the end of the day revolutionize industries?

July 17, 2024

by Frederic Jacquet

CORE

· 4,852 Views · 4 Likes

How To Convert Common Documents to PNG Image Arrays in Java

Learn how to convert Office documents, PDFs, and 100+ images to an array of PNG images, and review some of the benefits of making PNG conversions over other formats.

July 17, 2024

by Brian O'Neill

CORE

· 5,281 Views · 3 Likes

An Introduction to GraphQL: Simplifying Data Fetching With Real-World Examples

Understanding the basics of GraphQL: How to define schemas, set up a server, and integrate with front-end applications for efficient data management.

July 17, 2024

by Nitesh Upadhyaya

CORE

· 4,292 Views · 4 Likes

Why the Newest LLMs Use a MoE (Mixture of Experts) Architecture

Learn about the Mixture of Experts (MoE) architecture, defined as a mix or blend of different "expert" models working together to complete a specific problem.

July 17, 2024

by Kevin Vu

· 5,489 Views · 3 Likes

When Not to Choose Google Apache Kafka for BigQuery

When not to choose Google Apache Kafka for BigQuery — a review, comparison with Confluent Cloud, and data streaming landscape outlook.

July 16, 2024

by Kai Wähner

CORE

· 4,368 Views · 1 Like

Oracle SQL: Understanding Execution Plan and Performance Tuning

In this article, we will explore how each database generates and utilizes "explain" plans, highlighting the strengths and potential pitfalls of their approaches.

July 16, 2024

by Raosaheb Ghule

· 4,998 Views · 7 Likes

Understanding Big O Notation in Python

Learning Big O notation is crucial because it helps us understand the current efficiency of our code and guides us in optimizing it further.

July 16, 2024

by Shital Kat

· 3,176 Views · 1 Like

API Versioning in Microservices Architecture

This article explores what API Versioning is, why it is important for software development, and its benefits in microservices architecture and cloud computing.

July 16, 2024

by Gaurav Shekhar

· 6,190 Views · 2 Likes

Datafaker Gen: Leveraging BigQuery Sink on Google Cloud Platform

The simplest way to enrich a BigQuery table with realistic data. This guide describes how to define a schema for Datafaker Gen.

July 16, 2024

by Roman Rybak

· 4,945 Views · 2 Likes

Poisoning AI Brain: The Hidden Dangers of Third-Party Data and Agents in AI Systems

Backdoor attacks in LLMs involve hidden triggers embedded during training that activate harmful actions under specific conditions.

July 16, 2024

by Gaurav Puri

· 4,004 Views · 3 Likes

8 Ways AI Can Maximize the Value of Logs

Logs are filled with the information needed to monitor and understand systems. Let’s look at eight ways that AI can maximize the value of logs.

July 16, 2024

by Michael Bogan

CORE

· 4,142 Views · 2 Likes

Mastering DynamoDB: A Developer's Guide

Master DynamoDB's scalability, performance, and flexibility for building high-velocity, reliable, and fully managed applications.

July 15, 2024

by Lalithkumar Prakashchand

· 2,663 Views · 2 Likes

Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

Learn about a 10-fold improvement in startup performance, i.e., a 90% reduction in startup time, when using automatic checkpoint/restore at startup.

July 15, 2024

by Magnus Larsson

· 10,022 Views · 4 Likes

Questioning an Image Database With Local AI/LLM on Ollama and Spring AI

In this article, the reader will learn how to query an image database with natural language. Read further to learn more!

July 15, 2024

by Sven Loesekann

· 3,477 Views · 2 Likes

GBase 8s Database Locking Issues and Performance Optimization Strategies

This article provides an in-depth understanding of the locking situations in GBase 8s and offers a series of effective resolution strategies.

July 15, 2024

by Cong Li

· 1,260 Views · 1 Like

Alluxio Enterprise AI 3.2: Enhancing GPU Utilization and Data Access for AI Workloads

Alluxio Enterprise AI 3.2 revolutionizes GPU utilization and data access for AI workloads, offering seamless integration and enhanced performance.

July 15, 2024

by Tom Smith

CORE

· 2,518 Views · 1 Like

Factors to Consider: Data Warehouse Modernization

This article will help you with the factors that you will have to consider while choosing the new platform/endpoint, ETL suite, etc., for modernizing your data warehouse.

July 15, 2024

by Madhuri Veluri

· 3,116 Views · 1 Like

An Introduction To Open Table Formats

Path to performant, cost-effective, efficient data lakehouse enabling more intelligent interaction between storage and compute engines.

July 15, 2024

by Kranthi Kiran Erusu

· 3,808 Views · 5 Likes

Comparative Analysis of pgVector and OpenSearch for Vector Databases

This article compares pgVector and OpenSearch for vector databases, examining specifications, performance, and use cases.

July 14, 2024

by Jagadish Nimmagadda

· 4,709 Views · 1 Like

The Latest Data Engineering Topics