Data Engineering Resources

4 Reasons Agentic AI Is Reshaping Enterprise Search

Discover how Agentic AI is transforming enterprise search, empowering smarter decisions, and enhancing self-service success.

February 4, 2025

by Taranjeet Singh

· 3,665 Views · 2 Likes

Build Modern Data Architectures With Azure Data Services

Modern data architecture with Azure enables scalable, secure, and efficient data management, real-time analytics, and business innovation.

February 4, 2025

by Aravind Nuthalapati

CORE

· 35,599 Views · 1 Like

Limitations of LLM Reasoning

This article provides an analysis of the reasoning limitations that exist in today's LLMs and the approaches to overcome them.

February 4, 2025

by Bhala Ranganathan

CORE

· 6,767 Views · 3 Likes

Building a Full-Stack Resume Screening Application With AI

In this article, you'll learn how to build a smart resume analyzing application using the DeepSeek R1 open-source GenAI LLM model and save operation costs.

February 3, 2025

by Anujkumarsinh Donvir

CORE

· 10,056 Views · 9 Likes

Java Stream API: 3 Things Every Developer Should Know About

Java Stream API simplifies collection processing with lazy evaluation, parallelism, and functional programming. Use it to write cleaner, efficient, and scalable code.

February 3, 2025

by Danil Temnikov

CORE

· 15,700 Views · 16 Likes

Best Practices for API Rate Limits and Quotas

Rate limits protect your infrastructure, and quotas help you monetize your APIs. Both are key parts of a healthy API strategy.

February 3, 2025

by Derric Gilling

CORE

· 6,804 Views · 1 Like

Managing Distributed System Locks With Azure Storage

Locks in distributed systems give processes exclusive access to resources. Timeout and lease-based locks help prevent deadlocks and resource contention.

February 3, 2025

by Siri Varma Vegiraju

CORE

· 11,381 Views · 1 Like

All You Need to Know About Apache Spark

Apache Spark is a fast, open-source cluster computing framework for big data, supporting ML, SQL, and streaming. It’s scalable, efficient, and widely used.

February 3, 2025

by Abhishek Trehan

· 4,313 Views · 2 Likes

90% Cost Reduction With Prefix Caching for LLMs

Up to 70% of prompts in LLM applications are repetitive. Prefix caching can reduce inference costs by up to 90%, thus optimizing performance and saving money.

February 3, 2025

by Mahak Shah

· 4,839 Views

Building Neural Networks With Automatic Differentiation

In this post, we will write a basic DNN using simple Python. To do that, we need to understand automatic differentiation and then implement it in code.

February 3, 2025

by Mayank Gupta

· 2,983 Views · 2 Likes

Building RAG Apps With Apache Cassandra, Python, and Ollama

A brief introduction to Apache Cassandra for retrieval-augmented generation using Python and Ollama for developing applications free of cost locally or on a server.

February 3, 2025

by Varun Setia

· 4,515 Views

Pydantic: Simplifying Data Validation in Python

Pydantic is a powerful Python library that uses type annotations to validate data structures. Learn about the powerful features of Pydantic with code examples.

February 3, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 3,473 Views · 6 Likes

The Quest for HA and DR in Loki

Minimize data loss and business disruption by implementing high availability and configuring disaster recovery for Loki with AWS S3 as the object store.

January 31, 2025

by Pavan N G

· 4,175 Views · 1 Like

Building Custom Tools With Model Context Protocol

Learn how to build MCP servers to extend AI capabilities. Create tools that AI models can seamlessly integrate, demonstrated through an arXiv paper search implementation.

January 31, 2025

by Aditya Karnam Gururaj Rao

· 10,609 Views · 4 Likes

Web Scraping With LLMs, ScrapeGraphAI, and LangChain

In this article, learn how to use LLMs for web scraping with ScrapeGraphAI, LangChain, and Pydantic. This guide covers setup, configuration, and data extraction

January 31, 2025

by Juveria dalvi

· 4,931 Views

Creating a Service for Sensitive Data With Spring and Redis

In some cases, one cannot store user-sensitive data permanently. Let's create a simple application that handles sensitive data leveraging Spring and Redis.

January 31, 2025

by Alexander Rumyantsev

· 5,052 Views · 2 Likes

CAP and PACELC Theorems in Plain English

Data engineering fundamentals explained: the CAP theorem and PACELC — two frameworks for understanding tradeoffs in distributed systems.

January 31, 2025

by luminousmen _

· 2,701 Views

How to Build a Data Dashboard Prototype With Generative AI

This article is a tutorial that shows how to build a data dashboard to visualize book reading data taken from goodreads.com.

January 30, 2025

by Jo Stichbury

CORE

· 4,535 Views · 1 Like

Develop Microservices Using Azure Functions, API Management

Explore how to easily build cloud-deployed microservices using serverless technology with Azure Functions and connect with APIs standardized in API Management.

January 30, 2025

by Ankit Masrani

· 7,252 Views · 3 Likes

Building a Machine Learning Pipeline Using PySpark

This article discusses building an efficient ML pipeline with PySpark, covering data loading, preprocessing, model training, and evaluation for large datasets.

January 30, 2025

by Abhishek Trehan

· 5,497 Views · 1 Like

The Latest Data Engineering Topics