Data Engineering Resources

Building Custom Tools With Model Context Protocol

Learn how to build MCP servers to extend AI capabilities. Create tools that AI models can seamlessly integrate, demonstrated through an arXiv paper search implementation.

January 31, 2025

by Aditya Karnam Gururaj Rao

· 10,784 Views · 4 Likes

Web Scraping With LLMs, ScrapeGraphAI, and LangChain

In this article, learn how to use LLMs for web scraping with ScrapeGraphAI, LangChain, and Pydantic. This guide covers setup, configuration, and data extraction

January 31, 2025

by Juveria dalvi

· 4,971 Views

Creating a Service for Sensitive Data With Spring and Redis

In some cases, one cannot store user-sensitive data permanently. Let's create a simple application that handles sensitive data leveraging Spring and Redis.

January 31, 2025

by Alexander Rumyantsev

· 5,084 Views · 2 Likes

CAP and PACELC Theorems in Plain English

Data engineering fundamentals explained: the CAP theorem and PACELC — two frameworks for understanding tradeoffs in distributed systems.

January 31, 2025

by luminousmen _

· 2,743 Views

How to Build a Data Dashboard Prototype With Generative AI

This article is a tutorial that shows how to build a data dashboard to visualize book reading data taken from goodreads.com.

January 30, 2025

by Jo Stichbury

CORE

· 4,583 Views · 1 Like

Develop Microservices Using Azure Functions, API Management

Explore how to easily build cloud-deployed microservices using serverless technology with Azure Functions and connect with APIs standardized in API Management.

January 30, 2025

by Ankit Masrani

· 7,316 Views · 3 Likes

Building a Machine Learning Pipeline Using PySpark

This article discusses building an efficient ML pipeline with PySpark, covering data loading, preprocessing, model training, and evaluation for large datasets.

January 30, 2025

by Abhishek Trehan

· 5,759 Views · 1 Like

Bridging Graphviz and Cytoscape.js for Interactive Graphs

Making Graphviz static digraphs interactive and compatible with Cytoscape by converting DOT format graphs into Cytoscape JSON using Python.

January 30, 2025

by Puneet Malhotra

· 4,460 Views · 1 Like

SmartXML: An Alternative to XPath for Complex XML Files

We'll discuss SmartXML, an XPath alternative for parsing complex XML files, converting them to SQL, and loading the results into a database seamlessly.

January 30, 2025

by Luca Sanders

· 5,808 Views · 2 Likes

Commonly Occurring Errors in Microsoft Graph Integrations and How To Troubleshoot Them (Part 7)

This article discusses retrieving attachments from various SharePoint lists using the SharePoint API and Microsoft Graph API.

January 30, 2025

by Constantin Kwiatkowski

· 4,645 Views

Scaling Read Your Own Writes Consistency

This article is intended for distributed systems practitioners looking to understand and implement Read Your Own Writes consistency in production environments.

January 30, 2025

by Ganapathy Subramanian Ramachandran

· 2,928 Views

Passing JSON Variables in Azure Pipelines

Learn how to handle JSON variables in Azure DevOps pipelines, avoid escaping issues, and ensure seamless API integration with proper normalization techniques.

January 29, 2025

by Mohammed Basil

· 8,310 Views · 2 Likes

Predicting Diabetes Types: A Deep Learning Approach

Machine learning analysis of diabetes in young Indians: Deep learning vs. XGBoost (64.75% vs. 74% accuracy) using health and lifestyle data.

January 29, 2025

by Aditya Karnam Gururaj Rao

· 3,316 Views · 1 Like

Metal and the Simulated Annealing Algorithm

The Simulated Annealing algorithm described in this article demonstrates its effectiveness as a powerful tool for finding optimal solutions to complex problems.

January 29, 2025

by Vitaly Kuznetsov (Ippolitov)

· 3,325 Views · 2 Likes

Expert Guide: How to Slash Cloud Cost in 2025

In this guide, we will zero in on high-impact tips for cloud cost optimization. Cost is still a hurdle as many companies onboard to cloud computing.

January 29, 2025

by Milavkumar Shah

· 5,859 Views · 1 Like

Gemini 2.0 Flash (Experimental): A Deep Dive for Developers

Gemini 2.0 Flash — In the blog, let us see the top 10 abilities of the LLM and how developers can utilize it to develop various applications.

January 29, 2025

by Akash Lomas

· 4,148 Views · 1 Like

Scrape Amazon Product Reviews With Python

Let's learn how we can implement Python and Python scripts to scrape the Amazon website in an ethical way to extract product review data.

January 29, 2025

by Juveria dalvi

· 2,877 Views · 2 Likes

Implement RAG With PGVector, LangChain4j, and Ollama

Apply vector search and RAG experiments to enhance query results and optimize data storage for text embeddings, specifically with Bruce Springsteen's album data.

January 28, 2025

by Gunter Rotsaert

CORE

· 3,131 Views · 2 Likes