Data Engineering Resources

Agentic AI 101: Understanding Artificial Intelligence Agents

What is agentic AI and how do these autonomous systems improve productivity, facilitate smarter decisions, and at the end of the day revolutionize industries?

July 17, 2024

by Frederic Jacquet

CORE

· 8,849 Views · 5 Likes

How To Convert Common Documents to PNG Image Arrays in Java

Learn how to convert Office documents, PDFs, and 100+ images to an array of PNG images, and review some of the benefits of making PNG conversions over other formats.

July 17, 2024

by Brian O'Neill

CORE

· 6,804 Views · 3 Likes

An Introduction to GraphQL: Simplifying Data Fetching With Real-World Examples

Understanding the basics of GraphQL: How to define schemas, set up a server, and integrate with front-end applications for efficient data management.

July 17, 2024

by Nitesh Upadhyaya

CORE

· 5,505 Views · 4 Likes

Why the Newest LLMs Use a MoE (Mixture of Experts) Architecture

Learn about the Mixture of Experts (MoE) architecture, defined as a mix or blend of different "expert" models working together to complete a specific problem.

July 17, 2024

by Kevin Vu

· 6,617 Views · 3 Likes

When Not to Choose Google Apache Kafka for BigQuery

When not to choose Google Apache Kafka for BigQuery — a review, comparison with Confluent Cloud, and data streaming landscape outlook.

July 16, 2024

by Kai Wähner

CORE

· 5,255 Views · 1 Like

Oracle SQL: Understanding Execution Plan and Performance Tuning

In this article, we will explore how each database generates and utilizes "explain" plans, highlighting the strengths and potential pitfalls of their approaches.

July 16, 2024

by Raosaheb Ghule

· 20,552 Views · 8 Likes

Understanding Big O Notation in Python

Learning Big O notation is crucial because it helps us understand the current efficiency of our code and guides us in optimizing it further.

July 16, 2024

by Shital Kat

· 4,360 Views · 1 Like

API Versioning in Microservices Architecture

This article explores what API Versioning is, why it is important for software development, and its benefits in microservices architecture and cloud computing.

July 16, 2024

by Gaurav Shekhar

· 10,972 Views · 3 Likes

Datafaker Gen: Leveraging BigQuery Sink on Google Cloud Platform

The simplest way to enrich a BigQuery table with realistic data. This guide describes how to define a schema for Datafaker Gen.

July 16, 2024

by Roman Rybak

· 6,586 Views · 2 Likes

Poisoning AI Brain: The Hidden Dangers of Third-Party Data and Agents in AI Systems

Backdoor attacks in LLMs involve hidden triggers embedded during training that activate harmful actions under specific conditions.

July 16, 2024

by Gaurav Puri

· 5,061 Views · 3 Likes

8 Ways AI Can Maximize the Value of Logs

Logs are filled with the information needed to monitor and understand systems. Let’s look at eight ways that AI can maximize the value of logs.

July 16, 2024

by Michael Bogan

CORE

· 5,916 Views · 2 Likes

Mastering DynamoDB: A Developer's Guide

Master DynamoDB's scalability, performance, and flexibility for building high-velocity, reliable, and fully managed applications.

July 15, 2024

by Lalithkumar Prakashchand

· 3,991 Views · 3 Likes

Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

Learn about a 10-fold improvement in startup performance, i.e., a 90% reduction in startup time, when using automatic checkpoint/restore at startup.

July 15, 2024

by Magnus Larsson

· 11,780 Views · 6 Likes

Questioning an Image Database With Local AI/LLM on Ollama and Spring AI

In this article, the reader will learn how to query an image database with natural language. Read further to learn more!

July 15, 2024

by Sven Loesekann

· 4,753 Views · 2 Likes

Alluxio Enterprise AI 3.2: Enhancing GPU Utilization and Data Access for AI Workloads

Alluxio Enterprise AI 3.2 revolutionizes GPU utilization and data access for AI workloads, offering seamless integration and enhanced performance.

July 15, 2024

by Tom Smith

CORE

· 3,360 Views · 1 Like

Factors to Consider: Data Warehouse Modernization

This article will help you with the factors that you will have to consider while choosing the new platform/endpoint, ETL suite, etc., for modernizing your data warehouse.

July 15, 2024

by Madhuri Veluri

· 4,117 Views · 1 Like

An Introduction To Open Table Formats

Path to performant, cost-effective, efficient data lakehouse enabling more intelligent interaction between storage and compute engines.

July 15, 2024

by Kranthi Kiran Erusu

· 6,123 Views · 13 Likes

Comparative Analysis of pgVector and OpenSearch for Vector Databases

This article compares pgVector and OpenSearch for vector databases, examining specifications, performance, and use cases.

July 14, 2024

by Jagadish Nimmagadda

· 9,056 Views · 1 Like

7 Essential Tips for a Production ClickHouse Cluster

To set up a production cluster, use multiple replicas, allocate sufficient RAM, choose the right table engine, and limit the primary key to three columns.

July 12, 2024

by Vladimir Ivoninskii

· 10,671 Views · 3 Likes

Optimizing MySQL Performance: Best Practices for Database Efficiency

Optimize MySQL by tuning configurations, indexing effectively, performing regular maintenance, and utilizing advanced features like partitioning and replication.

July 12, 2024

by Lalithkumar Prakashchand

· 8,752 Views · 7 Likes

The Latest Data Engineering Topics