Data Engineering Resources

Optimizing Performance in Azure Cosmos DB: Best Practices and Tips

Optimize performance, minimize costs, and ensure scalability in Azure Cosmos DB with practices like selecting the right partition key and tuning queries.

January 2, 2025

by Muhammad Imran Ansari

· 12,835 Views

Avoiding If-Else: Advanced Approaches and Alternatives

Skip if-else statements by using design patterns, polymorphism, and functional strategies for cleaner, more adaptable, and significantly more testable code architectures.

January 2, 2025

by Taras Ivashchuk

· 11,713 Views · 14 Likes

Software Development Trends to Follow in 2025

In this blog, I will discuss key software development trends for 2025, from AI-powered tools to ethical AI, low-code platforms, and cloud-native advancements.

January 2, 2025

by Fawad Mohsin

· 8,787 Views · 7 Likes

Stop Being Afraid of Databases

Ensuring database reliability can be difficult. Our goal is to speed up development and minimize rollbacks. Achieving efficient processes requires database observability.

January 2, 2025

by Adam Furmanek

CORE

· 5,494 Views · 3 Likes

Kubernetes Deployments With DMZ Clusters: An Essential Guide

A DMZ cluster in Kubernetes secures public services from internal workloads, enhancing scalability, reducing attack surface, and ensuring controlled access.

January 2, 2025

by Sai Sandeep Ogety

CORE

· 9,845 Views · 13 Likes

Incremental Jobs and Data Quality Are On a Collision Course

Big data isn’t dead; it’s just going incremental. But bad things happen when uncontrolled changes collide with incremental jobs. Reacting to changes is a losing strategy.

January 1, 2025

by Jack Vanlightly

· 6,025 Views · 1 Like

Logical Reasoning in Network Problems

Online issues can be highly complex, requiring the capture of key information for a clear understanding. Through logical reasoning, the root cause was pinpointed.

January 1, 2025

by Bin Wang

· 7,607 Views · 4 Likes

Boosting Performance and Efficiency: Enhancing React Applications With GraphQL Over REST APIs

This article guides you through setting up a GraphQL server, defining schemas, and integrating with React, highlighting benefits and potential challenges.

January 1, 2025

by Shreyam Dutta Gupta

· 6,263 Views · 6 Likes

Driving RAG-Based AI Infrastructure

When paired with AI agents for workflow orchestration, RAG-based AI infrastructure excels in dynamic decision-making, analytics, and automation.

January 1, 2025

by Apurva Kumar

· 4,170 Views · 2 Likes

Streamlining Database Management: Running PostgreSQL in Docker Containers

You will learn how to run PostgreSQL in a Docker container with a portable, isolated, and resource-efficient database setup that is easy to manage and scalable.

December 31, 2024

by VARUNREDDY DEVIREDDY

· 6,990 Views · 3 Likes

Iceberg Catalogs: A Guide for Data Engineers

Learn about Apache Iceberg catalogs, their types, configurations, and the best-fit solutions for managing metadata and large datasets in different environments.

December 31, 2024

by Naresh Dulam

· 6,988 Views · 2 Likes

The Importance of Kubernetes in MLOps and Its Influence on Modern Businesses

The most critical challenges in MLOps involve constructing scalable and flexible infrastructure, to which Kubernetes is an efficient solution.

December 31, 2024

by Anupama Babu

· 5,744 Views · 1 Like

Enhancing DevOps in the Cloud With AI Innovation

Learn how AI-driven tools are transforming DevOps practices in cloud environments, improving automation, efficiency, and scalability by leveraging AI.

December 31, 2024

by Srinivas Reddy Bandarapu

· 4,756 Views · 3 Likes

Building a Receipt Scanner App With OCR, OpenAI, and PostgreSQL

In this article, you'll learn how to build a receipt scanner app using OCR for text extraction, OpenAI for processing, and PostgreSQL for efficient data storage.

December 31, 2024

by Balusamy Nachiappan

· 10,771 Views · 3 Likes

Agile Failure Has Strengthened, Not Weakened, Software

With psychological factors helping explain why humans struggle with Agile, interdisciplinary expertise can help save us from future software disasters.

December 31, 2024

by Junade Ali

· 5,120 Views · 2 Likes

Supercharge Your Coding Workflow With Ollama, LangChain, and RAG

Learn how to build a retrieval-augmented generation (RAG) code assistant using Ollama and LangChain to optimize your coding workflow.

December 30, 2024

by Santhosh Vijayabaskar

CORE

· 7,567 Views · 6 Likes

Setting Up a ScyllaDB Cluster on AWS Using Terraform

This article will teach you how to set up, deploy, manage, and scale ScyllaDB clusters on AWS using Terraform with this comprehensive step-by-step guide.

December 30, 2024

by Oleg Lipin

· 10,323 Views · 5 Likes

Bridging the Gap: Unlocking the Power of HDFS-Based Data Lakes With Streaming Databases

Big data technologies' quick development has highlighted the necessity of a smooth transition between real-time data analytics and batch processing systems.

December 30, 2024

by Gautam Goswami

CORE

· 3,168 Views · 1 Like

Getting Started With Snowflake Snowpark ML: A Step-by-Step Guide

This guide discusses setting up the Snowpark ML library, configuring your environment, and implementing a basic ML use case.

December 30, 2024

by Sevinthi Kali Sankar Nagarajan

· 4,179 Views · 2 Likes

Implementing CRUD Operations With NLP Using Microsoft.Extensions.AI

Learn how to implement CRUD operations using NLP in a .NET Web API application with the Microsoft.Extensions.AI library for a light management system.

December 30, 2024

by Baraneetharan Ramasamy

· 3,707 Views · 1 Like

The Latest Data Engineering Topics