Data Engineering Resources

Optimizing Trino Performance With Materialized Views in a Data Lake

In this article, learn how Trino materialized views boosted our Iceberg-based data lake, improving real-time query speed, reducing load, and cutting costs.

November 27, 2025

by Mikhail Povolotskii

· 1,476 Views

How to Push Docker Images to AWS Elastic Container Repository Using GitHub Actions

Learn how to automate CI/CD with GitHub Actions to build, tag, and push Docker images to AWS ECR, simplifying deployments directly from your repo.

November 26, 2025

by Purnima Upadhyaya

· 4,615 Views

Top 5 Practices for Building Dockerized MCP Servers

Learn the best practices for building MCP Servers and use them to power your LLM-powered applications. Make sure your setup has isolation and is secure.

November 26, 2025

by Mahak Shah

· 5,849 Views

Rethinking the Software Supply Chain for Agents

Today’s CI/CD pipelines aren’t built for AI. To make agentic systems reliable and trustworthy, we must evolve from continuous integration to continuous intelligence.

November 26, 2025

by Shibashis Mishra

· 1,756 Views · 2 Likes

Building a Local RAG App With a UI, No Vector DB Required

A step-by-step guide to building a complete retrieval-augmented generation (RAG) application with FAISS, LangChain, and Streamlit that runs 100% locally.

November 26, 2025

by Nabin Debnath

· 5,018 Views · 2 Likes

Building AI Agents With Semantic Kernel: A Practical 101 Guide

Learn how to build a simple, production-ready AI agent using Microsoft’s Semantic Kernel, covering kernels, plugins, agents, observability, and scalability.

November 26, 2025

by Sharan Babu Paramasivam Murugesan

· 2,108 Views · 2 Likes

LLMOps Under the Hood: Docker Practices for Large Language Model Deployment

LLMs need GPUs, libraries, and stable setups, which makes them hard to run. Docker simplifies this by packaging everything into portable containers.

November 26, 2025

by Pragya Keshap

· 3,629 Views

Vector Databases in Action: Building a RAG Pipeline for Code Search and Documentation

Build a semantic code search that understands meaning, not keywords, with AST parsing, embeddings, hybrid search, and LLM-powered documentation generation.

November 25, 2025

by Dinesh Elumalai

CORE

· 6,093 Views · 1 Like

Revamping Real-Time Data Ingestion for Scalable Media Intelligence

AI-augmented microservices streaming architecture (Spring Boot, Kafka, Elasticsearch) to handle 8.64 million real-time media articles per day.

November 25, 2025

by Sindhukumar Rajakumaran

· 1,309 Views

Integrating Lakeflow Connect With PostgreSQL: A Developer’s Complete Hands-On Guide From the Field

Traditional ingestion required custom ETL jobs that were costly to scale and maintain for PostgreSQL. To eliminate that overhead, I wired Lakeflow Connect for PostgreSQL.

November 25, 2025

by Jyoti Kunal Shah

· 8,518 Views

Effective Strategies for AWS Cost Optimization

Proven tips and techniques for optimizing AWS costs, including monitoring usage, setting budgets, and leveraging cost-effective services

November 25, 2025

by Prabhakar Mishra

· 2,541 Views · 1 Like

DevSecConflict: How Google Project Zero and FFmpeg Went Viral For All the Wrong Reasons

The tension between security and open source highlights the struggle over responsibility as AI uncovers vulnerabilities faster than we can respond.

November 24, 2025

by Katie Paxton-Fear

· 6,428 Views · 6 Likes

When Chatbots Go Rogue: Securing Conversational AI in Cyber Defense

Chatbots boost business but pose data risks. Strong security and AI risk management protect trust, compliance, and customer safety.

November 24, 2025

by Arun Goyal

· 2,721 Views · 1 Like

How to Build Real-Time Transaction Monitoring Systems With Streaming Data

Learn how to build real-time transaction monitoring systems with streaming data, exploring architecture, challenges, and best practices.

November 24, 2025

by Meet Deltan

· 3,023 Views · 1 Like

Building a Retrieval-Augmented Generation (RAG) System in Java With Spring AI, Vertex AI, and BigQuery

Build a Java RAG application using Spring Boot, Vertex AI embeddings, BigQuery vector search, and a web UI for interactive PDF-based question answering.

November 24, 2025

by Mohammed Fazalullah Qudrath

· 6,861 Views · 8 Likes

Creating an MCP Client With Spring AI

This blog post outlines the creation of an MCP client using Spring AI, building upon a previously established MCP server.

November 24, 2025

by Gunter Rotsaert

CORE

· 4,526 Views · 2 Likes

The Right to Be Forgotten in Event-Driven Data Products

Using event sourcing to propagate deletes across the data mesh for consistent retention and right-to-be-forgotten compliance.

November 24, 2025

by Murat Balkan

CORE

· 1,465 Views · 2 Likes

Architectural Evidence in Enterprise Java: Making Domain-Driven Design Visible

Use CDI stereotypes + JMolecules annotations to make DDD architecture explicit, enforceable, and testable. This preserves design intent as your Java system evolves.

November 24, 2025

by Otavio Santana

CORE

· 6,086 Views · 4 Likes