Data Engineering Resources

Dust Actors and Large Language Models: An Application

Use Dust Java Actors to create a pipeline that automatically finds, reads, and extracts specific info from news articles based on your topic of interest.

November 25, 2024

by Alan Littleford

· 5,034 Views

The Benefits of Using Cloud for Big Data Processing

Let's discuss the multiple advantages of using cloud computing for big data processing, from scalability to cost-effectiveness and enhanced collaboration.

November 25, 2024

by Job Ready Program

· 3,565 Views · 2 Likes

Implementing a RAG Model for PDF Content Extraction and Query Answering

Using Python to extract and process text from a PDF document, generate embeddings, calculate cosine similarity, and answer queries using the extracted content.

November 22, 2024

by Vaibhavi Tiwari

· 16,208 Views · 8 Likes

Micro-Frontends in a Microservice Architecture

Learn about how a microservice architecture based on self-contained systems with clean architecture is a useful compromise of different alternatives.

November 22, 2024

by Sven Loesekann

· 5,417 Views · 4 Likes

Knowledge Graphs and RAG: A Guide to AI Knowledge Retrieval

Enter knowledge graphs, the secret weapon for superior RAG applications. This guide has everything you need to begin leveraging RAG for intelligent AI knowledge retrieval.

Updated November 21, 2024

by Pavan Vemuri

CORE

· 9,564 Views · 13 Likes

Maximizing AI Training Efficiency: Selecting the Right Model

Unlock AI training efficiency: Learn to select the right model architecture for your task. Explore CNNs, RNNs, Transformers, and more to maximize performance.

November 21, 2024

by Kevin Vu

· 1,588 Views · 1 Like

Everyone Uses Postgres...But Why?

All big players in the market use PostgreSQL nowadays. But why is that? Find out the answer in this article.

November 20, 2024

by Adam Furmanek

CORE

· 8,081 Views · 13 Likes

How to Enable Azure Databricks Lakehouse Monitoring Through Scripts

This guide uses Python scripts to enable Databricks Lakehouse Monitoring for snapshot profiles for all Delta Live Tables in a schema in the Azure environment.

November 20, 2024

by Soumya Barman

· 2,419 Views · 1 Like

Deploy Smarter, Not Harder: A Guide to Setting Up AWS CI/CD Pipelines for ECS

Learn how to automate and optimize your container deployments with AWS tools for faster, scalable, and reliable application delivery.

November 20, 2024

by Manish Parapudi

· 1,858 Views

The Transformer Algorithm: A Love Story of Data and Attention

The Transformer algorithm revolutionized AI by using attention mechanisms to process data contextually and simultaneously, enhancing accuracy in tasks.

November 20, 2024

by Vasanthi Govindaraj

· 4,158 Views · 7 Likes

Building Intelligent Multi-Agent Systems With OpenAI’s Swarm

In this tutorial, we’ll use OpenAI’s Swarm to build a Smart Travel Concierge with collaborative AI agents. Learn the basics of agentic AI in this guide.

November 19, 2024

by Santhosh Vijayabaskar

CORE

· 5,173 Views · 8 Likes

Deployment Strategies for Apache Kafka Cluster Types

Multiple Kafka clusters enable hybrid integration, aggregation, migration, and disaster recovery across edge, data center, and multi-cloud environments.

November 19, 2024

by Kai Wähner

CORE

· 2,481 Views · 2 Likes

Finally, an ORM That Matches Modern Architectural Patterns!

How Stalactite ORM Matches Domain Purity, Single Responsibility Design, Separation of Concerns, and Clean Architecture Patterns

November 19, 2024

by Guillaume Mary

· 4,110 Views · 5 Likes

Top 10 PostgreSQL Tuning Tips for High-Performance Databases

This article provides 10 tips to optimize PostgreSQL performance, covering memory settings, query planning, indexing, autovacuum, and parallel queries.

November 18, 2024

by ELANGO MUTHUSAMY

· 6,198 Views · 13 Likes

A Beginner’s Guide: Build RAG With Milvus in 10 Minutes

Learn how to build your first RAG application with Milvus Lite and LangChain and turn vector embeddings into a naive RAG application using Python.

November 18, 2024

by Mert Bozkir

· 3,359 Views

The Role of Tokenization in LLMs: Does It Matter?

Tokenization breaks text into smaller parts (tokens) for LLMs to process and understand patterns efficiently. It’s essential for handling diverse languages.

November 18, 2024

by Sundeep Goud Katta

· 2,154 Views · 2 Likes

A Step-by-Step Guide to Building an MLOps Pipeline for LLMs and RAG

Learn how to build an automated MLOps pipeline for LLMs and RAG models, covering key aspects like training, deployment, and continuous performance monitoring.

November 18, 2024

by Kuppusamy Vellamadam Palavesam

· 3,157 Views · 2 Likes

How to Efficiently Manage Inactive (IDLE) Oracle Database Sessions

Learn how to manage inactive Oracle sessions using DCD, IDLE_TIME, and Resource Manager to optimize overall database performance and prevent resource issues.

November 18, 2024

by arvind toorpu

CORE

· 6,545 Views · 3 Likes

LangChain4J and Ollama: Chatbot Integration, Streaming, and RAG

Learn how to integrate LangChain4J and Ollama into your Java app and explore chatbot functionality, streaming, chat history, and retrieval-augmented generation.

November 15, 2024

by Nicolas Fränkel

· 2,726 Views · 12 Likes

Why AI Is a True System and More Than Just Software

AI is more than software — it's an adaptive system that combines various components to learn and interact, guided by human ethics.

November 14, 2024

by Frederic Jacquet

CORE

· 2,871 Views · 4 Likes

The Latest Data Engineering Topics