Data Engineering Resources

Learning Big O notation is crucial because it helps us understand the current efficiency of our code and guides us in optimizing it further.

July 16, 2024

by Shital Kat

· 4,233 Views · 1 Like

API Versioning in Microservices Architecture

This article explores what API Versioning is, why it is important for software development, and its benefits in microservices architecture and cloud computing.

July 16, 2024

by Gaurav Shekhar

· 10,658 Views · 3 Likes

Datafaker Gen: Leveraging BigQuery Sink on Google Cloud Platform

The simplest way to enrich a BigQuery table with realistic data. This guide describes how to define a schema for Datafaker Gen.

July 16, 2024

by Roman Rybak

· 6,449 Views · 2 Likes

Poisoning AI Brain: The Hidden Dangers of Third-Party Data and Agents in AI Systems

Backdoor attacks in LLMs involve hidden triggers embedded during training that activate harmful actions under specific conditions.

July 16, 2024

by Gaurav Puri

· 4,949 Views · 3 Likes

8 Ways AI Can Maximize the Value of Logs

Logs are filled with the information needed to monitor and understand systems. Let’s look at eight ways that AI can maximize the value of logs.

July 16, 2024

by Michael Bogan

CORE

· 5,654 Views · 2 Likes

Mastering DynamoDB: A Developer's Guide

Master DynamoDB's scalability, performance, and flexibility for building high-velocity, reliable, and fully managed applications.

July 15, 2024

by Lalithkumar Prakashchand

· 3,870 Views · 3 Likes

Faster Startup With Spring Boot 3.2 and CRaC, Part 1: Automatic Checkpoint

Learn about a 10-fold improvement in startup performance, i.e., a 90% reduction in startup time, when using automatic checkpoint/restore at startup.

July 15, 2024

by Magnus Larsson

· 11,646 Views · 6 Likes

Questioning an Image Database With Local AI/LLM on Ollama and Spring AI

In this article, the reader will learn how to query an image database with natural language. Read further to learn more!

July 15, 2024

by Sven Loesekann

· 4,650 Views · 2 Likes

Alluxio Enterprise AI 3.2: Enhancing GPU Utilization and Data Access for AI Workloads

Alluxio Enterprise AI 3.2 revolutionizes GPU utilization and data access for AI workloads, offering seamless integration and enhanced performance.

July 15, 2024

by Tom Smith

CORE

· 3,248 Views · 1 Like

Factors to Consider: Data Warehouse Modernization

This article will help you with the factors that you will have to consider while choosing the new platform/endpoint, ETL suite, etc., for modernizing your data warehouse.

July 15, 2024

by Madhuri Veluri

· 3,972 Views · 1 Like

An Introduction To Open Table Formats

Path to performant, cost-effective, efficient data lakehouse enabling more intelligent interaction between storage and compute engines.

July 15, 2024

by Kranthi Kiran Erusu

· 5,791 Views · 13 Likes

Comparative Analysis of pgVector and OpenSearch for Vector Databases

This article compares pgVector and OpenSearch for vector databases, examining specifications, performance, and use cases.

July 14, 2024

by Jagadish Nimmagadda

· 8,885 Views · 1 Like

7 Essential Tips for a Production ClickHouse Cluster

To set up a production cluster, use multiple replicas, allocate sufficient RAM, choose the right table engine, and limit the primary key to three columns.

July 12, 2024

by Vladimir Ivoninskii

· 10,425 Views · 3 Likes

Optimizing MySQL Performance: Best Practices for Database Efficiency

Optimize MySQL by tuning configurations, indexing effectively, performing regular maintenance, and utilizing advanced features like partitioning and replication.

July 12, 2024

by Lalithkumar Prakashchand

· 8,665 Views · 7 Likes

API Implementation on AWS Serverless Architecture

This article describes the implementation of rest API with AWS serverless infrastructure and explores the benefits of serverless over the traditional approach.

July 12, 2024

by Shailesh Hurdale

· 9,581 Views · 8 Likes

How To Reduce Memory Consumption in Integration Tests With Kafka Using GraalVM

This article details the process of creating a native image for EmbeddedKafka with GraalVM, highlighting its benefits in integration tests.

July 12, 2024

by Anton Belyaev

· 6,319 Views · 1 Like

AI: Do You Trust It?

Developing an AI-based app for generating weekly reports showed promising results, though complete trust in its reliability remains a challenge.

July 11, 2024

by Volodya Lombrozo

· 5,314 Views · 5 Likes

Create JWT Using DataWeave JWT Library

This article describes the process of creating JWT using the DataWeave JWT Library available in Mulesoft Exchange, which supports both HMAC and RSA algorithms.

July 11, 2024

by Ujala Kumar Yadav

· 9,569 Views · 2 Likes