Data Engineering Resources

Deploy Smarter, Not Harder: A Guide to Setting Up AWS CI/CD Pipelines for ECS

Learn how to automate and optimize your container deployments with AWS tools for faster, scalable, and reliable application delivery.

November 20, 2024

by Manish Parapudi

· 1,712 Views

The Transformer Algorithm: A Love Story of Data and Attention

The Transformer algorithm revolutionized AI by using attention mechanisms to process data contextually and simultaneously, enhancing accuracy in tasks.

November 20, 2024

by Vasanthi Govindaraj

· 3,476 Views · 7 Likes

Building Intelligent Multi-Agent Systems With OpenAI’s Swarm

In this tutorial, we’ll use OpenAI’s Swarm to build a Smart Travel Concierge with collaborative AI agents. Learn the basics of agentic AI in this guide.

November 19, 2024

by Santhosh Vijayabaskar

CORE

· 4,922 Views · 8 Likes

Deployment Strategies for Apache Kafka Cluster Types

Multiple Kafka clusters enable hybrid integration, aggregation, migration, and disaster recovery across edge, data center, and multi-cloud environments.

November 19, 2024

by Kai Wähner

CORE

· 2,343 Views · 2 Likes

Finally, an ORM That Matches Modern Architectural Patterns!

How Stalactite ORM Matches Domain Purity, Single Responsibility Design, Separation of Concerns, and Clean Architecture Patterns

November 19, 2024

by Guillaume Mary

· 3,952 Views · 5 Likes

Top 10 PostgreSQL Tuning Tips for High-Performance Databases

This article provides 10 tips to optimize PostgreSQL performance, covering memory settings, query planning, indexing, autovacuum, and parallel queries.

November 18, 2024

by ELANGO MUTHUSAMY

· 5,976 Views · 13 Likes

A Beginner’s Guide: Build RAG With Milvus in 10 Minutes

Learn how to build your first RAG application with Milvus Lite and LangChain and turn vector embeddings into a naive RAG application using Python.

November 18, 2024

by Mert Bozkir

· 3,119 Views

The Role of Tokenization in LLMs: Does It Matter?

Tokenization breaks text into smaller parts (tokens) for LLMs to process and understand patterns efficiently. It’s essential for handling diverse languages.

November 18, 2024

by Sundeep Goud Katta

· 1,944 Views · 2 Likes

A Step-by-Step Guide to Building an MLOps Pipeline for LLMs and RAG

Learn how to build an automated MLOps pipeline for LLMs and RAG models, covering key aspects like training, deployment, and continuous performance monitoring.

November 18, 2024

by Kuppusamy Vellamadam Palavesam

· 2,918 Views · 2 Likes

How to Efficiently Manage Inactive (IDLE) Oracle Database Sessions

Learn how to manage inactive Oracle sessions using DCD, IDLE_TIME, and Resource Manager to optimize overall database performance and prevent resource issues.

November 18, 2024

by arvind toorpu

CORE

· 5,912 Views · 3 Likes

LangChain4J and Ollama: Chatbot Integration, Streaming, and RAG

Learn how to integrate LangChain4J and Ollama into your Java app and explore chatbot functionality, streaming, chat history, and retrieval-augmented generation.

November 15, 2024

by Nicolas Fränkel

· 2,471 Views · 12 Likes

Why AI Is a True System and More Than Just Software

AI is more than software — it's an adaptive system that combines various components to learn and interact, guided by human ethics.

November 14, 2024

by Frederic Jacquet

CORE

· 2,667 Views · 4 Likes

Data Architectures in the AI Era: Key Strategies and Insights

Data architecture is evolving rapidly due to the rise of GenAI, requiring companies to move away from data silos toward integrated data fabrics and data meshes.

November 14, 2024

by Suri (thammuio)

CORE

· 24,446 Views · 5 Likes

The Science Behind Durability: Write-Ahead Logging Explained

For any persistence store system, guaranteeing durability of data being managed is of prime importance. Read on to know how write ahead logging ensures durability.

November 14, 2024

by Ammar Husain

CORE

· 2,070 Views · 2 Likes

Why Should Databases Go Natural?

From search to CRM, applications are adopting natural language and intuitive interactions. Should databases follow? This article provides a strategic perspective.

November 13, 2024

by Keshav Murthy

CORE

· 1,683 Views · 3 Likes

How to Get There: Bridging the Technology Gap Preventing You From Adopting a Secrets-Free Machine Identity Framework

Learn how to go from a world of secrets sprawl and into a future with secrets-free machine identity frameworks by adopting SPIFFE/SPIRE

November 13, 2024

by Dwayne McDaniel

· 1,926 Views

Building NLU Flows With Firebase GenKit and GitHub Models Powered by LLMs

Learn how to set up an NLU flow using Firebase Genkit AI and Firebase Functions to detect intents and entities from text using GitHub Models.

November 13, 2024

by Xavier Portilla Edo

CORE

· 891 Views · 2 Likes

Database Subsetting With Jailer

In this tutorial, you will learn how to export consistent sets of rows from relational databases to topologically sorted SQL-DML.

Updated November 13, 2024

by Ralf Wisser

· 16,606 Views · 3 Likes

Apache Iceberg: The Open Table Format for Lakehouses and Data Streaming

This article explores the table format wars of Apache Iceberg, Hudi, Delta Lake and XTable; and the product strategy of Snowflake, Databricks, Confluent, AWS, and Google.

November 12, 2024

by Kai Wähner

CORE

· 5,144 Views · 2 Likes

LLMops: The Future of AI Model Management

LLMOps enhances MLOps for generative AI, focusing on prompt and RAG management to boost efficiency, scalability, and streamline deployment while tackling resource and complexity challenges.

November 12, 2024

by Suri (thammuio)

CORE

· 28,316 Views · 1 Like

The Latest Data Engineering Topics