Data Engineering Resources

Search: From Basic Document Retrieval to Answer Generation

Exploring the evolution of document retrieval systems from traditional text-matching and frequency-based methods to advanced ingestion and retrieval strategies.

February 18, 2025

by Meghana Puvvadi

· 2,946 Views · 1 Like

The Slides of Hands-On Agile 2025

The virtual Hands-on Agile 2025 conference held last month discussed embracing the shift to context-based agility, from leadership to strategy to AI.

February 17, 2025

by Stefan Wolpers

CORE

· 5,855 Views · 1 Like

Integrating Apex With Lightning Web Components

Integrating your LWC with Apex to enable seamless communication between the frontend and backend, to provide robust data handling and processing.

February 17, 2025

by Jaseem Pookandy

CORE

· 4,514 Views · 2 Likes

Scaling ML Models Efficiently With Shared Neural Networks

A shared encoder architecture to decouple customer-specific fine-tuned models from the shared encoder to deploy at scale.

February 17, 2025

by Meghana Puvvadi

· 3,196 Views · 7 Likes

Creating a Web Project: Key Steps to Identify Issues

Learn how to identify web project issues through performance analysis, effective debugging, and product metrics tracking.

February 17, 2025

by Filipp Shcherbanich

CORE

· 6,036 Views · 4 Likes

Have LLMs Solved the Search Problem?

LLMs, while strong in content generation, need techniques like semantic chunking and vector embeddings to address the search problem in complex data environments.

February 17, 2025

by Avi Dubey

· 3,306 Views · 1 Like

Build a Data Analytics Platform With Flask, SQL, and Redis

Building a Flask-based web app that has dynamic querying for population thresholds, Redis caching for faster queries, and secure, scalable architecture.

February 17, 2025

by Sushma Kukkadapu

· 5,128 Views · 2 Likes

From Data to Decisions: Visualizing SAP Insights With Python

SAP data insights simplified with Python: A versatile tool for advanced analytics, automation, and managing large datasets.

February 17, 2025

by Prasanna Chandran Melnatami Krishnaram

· 2,601 Views · 5 Likes

Why Cloud Matters: Building Global, Scalable Microservices

A hands-on experience: Learn how to build a Book Management Catalog Microservice using Java, Helidon, Jakarta NoSQL, and Oracle Cloud.

February 17, 2025

by Otavio Santana

CORE

· 3,920 Views · 5 Likes

Redis as a Primary Database for Complex Applications

This article aims to answer one question: How can Redis be used as a primary database for complex applications that need to store data in multiple formats?

February 14, 2025

by Mohammed Talib

· 7,204 Views · 4 Likes

Optimizing Database Performance in Middleware Applications

Optimize database interactions in middleware with caching, parallelism, write batching, and connection pooling to boost performance and scalability effectively.

February 14, 2025

by karandeep johar

· 6,902 Views · 5 Likes

ETL Generation Using GenAI

Learn about how GenAI automates ETL pipelines, generates code, adapts to schema changes, and improves data processes with speed, efficiency, and precision.

February 14, 2025

by Ramesh Daddala

· 6,493 Views · 2 Likes

Unlocking Local AI: Build RAG Apps Without Cloud or API Keys

In this tutorial, we will use Chipper, an open-source framework that simplifies building local RAG applications without cloud dependencies or API keys.

February 14, 2025

by Santhosh Vijayabaskar

CORE

· 6,314 Views · 3 Likes

Git Tags vs AWS Tags: A Tag-Tastic Showdown!

The ultimate showdown between Git Tags and AWS Tags, two contenders with nothing in common except their love for the word "tag." Let's see them go head-to-head.

February 14, 2025

by Raghava Dittakavi

CORE

· 4,729 Views · 1 Like

AI Summarization: Extractive and Abstractive Techniques

Learn how to create an AI-powered summarization tool using Hugging Face and OpenAI, combining extractive and abstractive methods for concise, accurate results.

February 13, 2025

by Vaibhavi Tiwari

· 4,247 Views · 1 Like

How to Backup SQL Server RDS to an S3 Bucket

Learn how to back up your SQL Server RDS database directly to an Amazon S3 bucket using msdb.dbo.rds_backup_database, with detailed examples.

February 13, 2025

by arvind toorpu

CORE

· 5,382 Views · 3 Likes

Idempotency in Distributed Systems: When and Why It Matters

This article explains idempotency in distributed systems and ensuring consistent results regardless of multiple executions, with implementation and challenges.

February 13, 2025

by Sandeep Kumar Gond

· 5,342 Views

Goose Migrations for Smooth Database Changes

Migrate safely: Goose automates DB changes, version control, and rollbacks. Minimal downtime — ideal for CI/CD and high load.

February 12, 2025

by Ilia Ivankin

· 9,836 Views

Loading XML into MongoDB

Learn how to export XML data to MongoDB using SmartXML ETL tools, simplifying the process and ensuring efficient data handling and storage.

February 12, 2025

by Luca Sanders

· 6,847 Views · 1 Like

The Right ETL Architecture for Multi-Source Data Integration

Dedicated ETL pipelines are easy to set up but hard to scale, while common pipelines offer efficiency at the cost of complexity. Know which one to choose.

February 12, 2025

by Murat Balkan

CORE

· 6,178 Views · 1 Like

The Latest Data Engineering Topics