Data Engineering Resources

Container Checkpointing in Kubernetes With a Custom API

This article discusses using a Kubernetes sidecar for container checkpointing: build, push, deploy to K8s, and trigger checkpoints via API for state management.

February 18, 2025

by VARUNREDDY DEVIREDDY

· 5,860 Views · 1 Like

The Delegated Chain of Thought Architecture

The D-CoT architecture decouples reasoning from execution in LLM by centralizing reasoning in a "modulith" and delegating execution tasks to specialized modules.

February 18, 2025

by Pier-Jean MALANDRINO

CORE

· 4,615 Views · 4 Likes

Generate Unit Tests With AI Using Ollama and Spring Boot

Run Ollama LLM locally with Spring Boot to generate unit tests for Java. Ensure data privacy, easy setup, and seamless integration with Spring AI.

February 18, 2025

by Amol Gote

CORE

· 8,101 Views · 4 Likes

Multimodal RAG With Colpali, Milvus, and VLMs

Build a multimodal RAG app with ColPali, Milvus, and a visual language model to enable Q&A on PDFs using text and visual data indexed for efficient search.

February 18, 2025

by Saumitra Srivastav

· 1,967 Views · 3 Likes

Search: From Basic Document Retrieval to Answer Generation

Exploring the evolution of document retrieval systems from traditional text-matching and frequency-based methods to advanced ingestion and retrieval strategies.

February 18, 2025

by Meghana Puvvadi

· 2,876 Views · 1 Like

The Slides of Hands-On Agile 2025

The virtual Hands-on Agile 2025 conference held last month discussed embracing the shift to context-based agility, from leadership to strategy to AI.

February 17, 2025

by Stefan Wolpers

CORE

· 5,807 Views · 1 Like

Integrating Apex With Lightning Web Components

Integrating your LWC with Apex to enable seamless communication between the frontend and backend, to provide robust data handling and processing.

February 17, 2025

by Jaseem Pookandy

CORE

· 4,288 Views · 2 Likes

Scaling ML Models Efficiently With Shared Neural Networks

A shared encoder architecture to decouple customer-specific fine-tuned models from the shared encoder to deploy at scale.

February 17, 2025

by Meghana Puvvadi

· 3,098 Views · 7 Likes

Creating a Web Project: Key Steps to Identify Issues

Learn how to identify web project issues through performance analysis, effective debugging, and product metrics tracking.

February 17, 2025

by Filipp Shcherbanich

CORE

· 5,971 Views · 4 Likes

Have LLMs Solved the Search Problem?

LLMs, while strong in content generation, need techniques like semantic chunking and vector embeddings to address the search problem in complex data environments.

February 17, 2025

by Avi Dubey

· 3,173 Views · 1 Like

Build a Data Analytics Platform With Flask, SQL, and Redis

Building a Flask-based web app that has dynamic querying for population thresholds, Redis caching for faster queries, and secure, scalable architecture.

February 17, 2025

by Sushma Kukkadapu

· 4,902 Views · 2 Likes

From Data to Decisions: Visualizing SAP Insights With Python

SAP data insights simplified with Python: A versatile tool for advanced analytics, automation, and managing large datasets.

February 17, 2025

by Prasanna Chandran Melnatami Krishnaram

· 2,501 Views · 5 Likes

Why Cloud Matters: Building Global, Scalable Microservices

A hands-on experience: Learn how to build a Book Management Catalog Microservice using Java, Helidon, Jakarta NoSQL, and Oracle Cloud.

February 17, 2025

by Otavio Santana

CORE

· 3,840 Views · 5 Likes

Redis as a Primary Database for Complex Applications

This article aims to answer one question: How can Redis be used as a primary database for complex applications that need to store data in multiple formats?

February 14, 2025

by Mohammed Talib

· 7,138 Views · 4 Likes

Optimizing Database Performance in Middleware Applications

Optimize database interactions in middleware with caching, parallelism, write batching, and connection pooling to boost performance and scalability effectively.

February 14, 2025

by karandeep johar

· 6,772 Views · 5 Likes

ETL Generation Using GenAI

Learn about how GenAI automates ETL pipelines, generates code, adapts to schema changes, and improves data processes with speed, efficiency, and precision.

February 14, 2025

by Ramesh Daddala

· 6,384 Views · 2 Likes

Unlocking Local AI: Build RAG Apps Without Cloud or API Keys

In this tutorial, we will use Chipper, an open-source framework that simplifies building local RAG applications without cloud dependencies or API keys.

February 14, 2025

by Santhosh Vijayabaskar

CORE

· 6,156 Views · 3 Likes

Git Tags vs AWS Tags: A Tag-Tastic Showdown!

The ultimate showdown between Git Tags and AWS Tags, two contenders with nothing in common except their love for the word "tag." Let's see them go head-to-head.

February 14, 2025

by Raghava Dittakavi

CORE

· 4,636 Views · 1 Like

AI Summarization: Extractive and Abstractive Techniques

Learn how to create an AI-powered summarization tool using Hugging Face and OpenAI, combining extractive and abstractive methods for concise, accurate results.

February 13, 2025

by Vaibhavi Tiwari

· 4,114 Views · 1 Like

How to Backup SQL Server RDS to an S3 Bucket

Learn how to back up your SQL Server RDS database directly to an Amazon S3 bucket using msdb.dbo.rds_backup_database, with detailed examples.

February 13, 2025

by arvind toorpu

CORE

· 5,261 Views · 3 Likes

The Latest Data Engineering Topics