Data Engineering Resources

An Introduction to GraphQL: Simplifying Data Fetching With Real-World Examples

Understanding the basics of GraphQL: How to define schemas, set up a server, and integrate with front-end applications for efficient data management.

July 17, 2024

by Nitesh Upadhyaya

CORE

· 5,654 Views · 4 Likes

Why the Newest LLMs Use a MoE (Mixture of Experts) Architecture

Learn about the Mixture of Experts (MoE) architecture, defined as a mix or blend of different "expert" models working together to complete a specific problem.

July 17, 2024

by Kevin Vu

· 6,835 Views · 3 Likes

When Not to Choose Google Apache Kafka for BigQuery

When not to choose Google Apache Kafka for BigQuery — a review, comparison with Confluent Cloud, and data streaming landscape outlook.

July 16, 2024

by Kai Wähner

CORE

· 5,343 Views · 2 Likes

Oracle SQL: Understanding Execution Plan and Performance Tuning

In this article, we will explore how each database generates and utilizes "explain" plans, highlighting the strengths and potential pitfalls of their approaches.

July 16, 2024

by Raosaheb Ghule

· 20,969 Views · 8 Likes

Understanding Big O Notation in Python

Learning Big O notation is crucial because it helps us understand the current efficiency of our code and guides us in optimizing it further.

July 16, 2024

by Shital Kat

· 4,528 Views · 1 Like

API Versioning in Microservices Architecture

This article explores what API Versioning is, why it is important for software development, and its benefits in microservices architecture and cloud computing.

July 16, 2024

by Gaurav Shekhar

· 11,217 Views · 3 Likes

Datafaker Gen: Leveraging BigQuery Sink on Google Cloud Platform

The simplest way to enrich a BigQuery table with realistic data. This guide describes how to define a schema for Datafaker Gen.

July 16, 2024

by Roman Rybak

· 6,740 Views · 2 Likes

Poisoning AI Brain: The Hidden Dangers of Third-Party Data and Agents in AI Systems

Backdoor attacks in LLMs involve hidden triggers embedded during training that activate harmful actions under specific conditions.

July 16, 2024

by Gaurav Puri

· 5,221 Views · 3 Likes

8 Ways AI Can Maximize the Value of Logs

Logs are filled with the information needed to monitor and understand systems. Let’s look at eight ways that AI can maximize the value of logs.

July 16, 2024

by Michael Bogan

CORE

· 6,058 Views · 2 Likes

Mastering DynamoDB: A Developer's Guide

Master DynamoDB's scalability, performance, and flexibility for building high-velocity, reliable, and fully managed applications.

July 15, 2024

by Lalithkumar Prakashchand

· 4,111 Views · 3 Likes

Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

Learn about a 10-fold improvement in startup performance, i.e., a 90% reduction in startup time, when using automatic checkpoint/restore at startup.

July 15, 2024

by Magnus Larsson

· 11,926 Views · 6 Likes

Questioning an Image Database With Local AI/LLM on Ollama and Spring AI

In this article, the reader will learn how to query an image database with natural language. Read further to learn more!

July 15, 2024

by Sven Loesekann

· 4,893 Views · 2 Likes

Alluxio Enterprise AI 3.2: Enhancing GPU Utilization and Data Access for AI Workloads

Alluxio Enterprise AI 3.2 revolutionizes GPU utilization and data access for AI workloads, offering seamless integration and enhanced performance.

July 15, 2024

by Tom Smith

CORE

· 3,488 Views · 2 Likes

Factors to Consider: Data Warehouse Modernization

This article will help you with the factors that you will have to consider while choosing the new platform/endpoint, ETL suite, etc., for modernizing your data warehouse.

July 15, 2024

by Madhuri Veluri

· 4,220 Views · 2 Likes

An Introduction To Open Table Formats

Path to performant, cost-effective, efficient data lakehouse enabling more intelligent interaction between storage and compute engines.

July 15, 2024

by Kranthi Kiran Erusu

· 6,328 Views · 14 Likes

Comparative Analysis of pgVector and OpenSearch for Vector Databases

This article compares pgVector and OpenSearch for vector databases, examining specifications, performance, and use cases.

July 14, 2024

by Jagadish Nimmagadda

· 9,189 Views · 1 Like

7 Essential Tips for a Production ClickHouse Cluster

To set up a production cluster, use multiple replicas, allocate sufficient RAM, choose the right table engine, and limit the primary key to three columns.

July 12, 2024

by Vladimir Ivoninskii

· 10,930 Views · 3 Likes

Optimizing MySQL Performance: Best Practices for Database Efficiency

Optimize MySQL by tuning configurations, indexing effectively, performing regular maintenance, and utilizing advanced features like partitioning and replication.

July 12, 2024

by Lalithkumar Prakashchand

· 8,879 Views · 7 Likes

API Implementation on AWS Serverless Architecture

This article describes the implementation of rest API with AWS serverless infrastructure and explores the benefits of serverless over the traditional approach.

July 12, 2024

by Shailesh Hurdale

· 9,910 Views · 8 Likes

How To Reduce Memory Consumption in Integration Tests With Kafka Using GraalVM

This article details the process of creating a native image for EmbeddedKafka with GraalVM, highlighting its benefits in integration tests.

July 12, 2024

by Anton Belyaev

· 6,551 Views · 2 Likes

The Latest Data Engineering Topics