Data Engineering Resources

Databricks 101: An Introductory Guide on Navigating and Optimizing This Data Powerhouse

Gain insight into Databrick’s workspace, the pivotal role of workflows, and the nuanced world of compute resources.

December 12, 2024

by Noa Shavit

· 4,818 Views · 1 Like

Idempotency and Reliability in Event-Driven Systems: A Practical Guide

In this guide, learn how implementing robust idempotent patterns and leveraging tools can mitigate the risks posed by retries and duplicate events.

December 12, 2024

by Vikram Mohanagandhi

· 6,418 Views · 2 Likes

DevOps Tutorial: Docker, Kubernetes, and Azure DevOps

In this article, we break down everything you need to know about DevOps, so that you can get started building your own CI/CD pipeline.

Updated December 11, 2024

by Ranga Karanam

CORE

· 42,770 Views · 18 Likes

How and Why the Developer-First Approach Is Changing the Observability Landscape

Developers play a crucial role in modern companies. Explore more about the need to have a developer-first approach and include observability from day one.

December 11, 2024

by Adam Furmanek

CORE

· 5,652 Views · 3 Likes

How to Test PUT Requests for API Testing With Playwright Java

This tutorial demonstrates how to test PUT requests using the Playwright Java framework for API testing, including examples of updating data of an API.

December 11, 2024

by Faisal Khatri

CORE

· 9,663 Views · 2 Likes

Data Processing With Python: Choosing Between MPI and Spark

Know the differences between MPI and Spark for big data processing and find out which framework suits your needs for parallel and distributed computing.

December 11, 2024

by Anil Kumar Moka

· 4,794 Views

DuckDB Optimization: A Developer's Guide to Better Performance

DuckDB is a powerful in-memory database that offers parallel processing capabilities and this guide will outline optimization techniques to get the most out of it.

December 11, 2024

by Anil Kumar Moka

· 5,289 Views · 4 Likes

Setting Up DBT and Snowpark for Machine Learning Pipelines

This guide describes how to install and set up DBT and Snowpark for machine-learning model pipelines to streamline data processing and feature engineering tasks.

December 11, 2024

by Sevinthi Kali Sankar Nagarajan

· 7,076 Views · 3 Likes

Duration Bias in Video Recommendations: A Complete Guide to Fair Content Ranking

Explore the theoretical foundations and practical strategies for addressing duration bias to create balanced, fair, and effective recommendation systems.

December 11, 2024

by Amey Porobo Dharwadker

· 2,934 Views · 2 Likes

Generic and Dynamic API: MuleSoft

This article demonstrates the definition of a generic and dynamic data model that can be used for various publisher-generated events.

December 11, 2024

by Ankur Bhuyan

· 3,601 Views · 2 Likes

Protecting Your API Ecosystem: The Role of Rate Limiting in Service Stability

Explore effective API rate limiting strategies to safeguard against overloading, promote fair usage, and improve performance.

December 10, 2024

by Elakkiya Daivam

· 5,193 Views · 4 Likes

Advancing Explainable Natural Language Generation (NLG): Techniques, Challenges, and Applications

Incorporating XAI techniques helps developers refine models, uncover biases, and ensure reliable and fair applications of NLG.

December 10, 2024

by Manasi Sharma

· 3,473 Views · 2 Likes

Understanding Functional Programming: A Quick Guide for Beginners

This quick and simple guide to functional programming concepts covers pure functions, immutability, and declarative coding for cleaner, predictable code.

December 10, 2024

by Sri Harshitha Yarra

· 8,545 Views · 7 Likes

A Practical Guide to Securing NodeJS APIs With JWT

JWT secures your Node.js/Express APIs statelessly, ensuring each request is authenticated with a lightweight, tamper-proof token for scalable, modern security.

December 10, 2024

by Anujkumarsinh Donvir

CORE

· 7,283 Views · 11 Likes

How to Automate Blob Deletion in Azure Storage Using PowerShell

Learn how to automate daily blob deletion in Azure storage accounts using PowerShell, effectively manage timestamped reports, and optimize storage efficiency.

December 10, 2024

by thiyagu selvaraj

· 6,711 Views · 1 Like

In-Depth Analysis of MySQL 8.0 Performance Degradation

We've analyzed the causes of increased response latency in MySQL 8.0. Users can understand the performance decline from both read/write operations and the code level.

December 10, 2024

by Bin Wang

· 4,040 Views · 2 Likes

Demystifying Kubernetes in 5 Minutes

Take a few minutes to learn about Kubernetes: a powerful container orchestration platform that simplifies the management of applications.

December 10, 2024

by Samarth Shah

· 8,779 Views · 6 Likes

Why I'll Never Go Back to GZIP After Trying ZSTD

Faster speeds. Better compression. Facebook's ZSTD is rapidly becoming the new standard for data compression in modern applications.

December 10, 2024

by Aditya Karnam Gururaj Rao

· 2,651 Views

Enhancing GenAI Applications With KubeMQ: Efficiently Scaling Retrieval-Augmented Generation

Achieve efficiency and reliability in your GenAI RAG workflows with KubeMQ for seamless message handling and FalkorDB for fast, scalable data storage/retrieval.

December 10, 2024

by Alvin Lee

CORE

· 2,516 Views · 3 Likes

Chunking Strategies for Optimizing Large Language Models (LLMs)

Different chunking approaches, such as fixed-size, recursive, semantic, and agentic chunking, are discussed, each with its own advantages.

December 9, 2024

by Usama Jamil

· 2,721 Views · 2 Likes

The Latest Data Engineering Topics