Data Engineering Resources

Agile Failure Has Strengthened, Not Weakened, Software

With psychological factors helping explain why humans struggle with Agile, interdisciplinary expertise can help save us from future software disasters.

December 31, 2024

by Junade Ali

· 5,081 Views · 2 Likes

Supercharge Your Coding Workflow With Ollama, LangChain, and RAG

Learn how to build a retrieval-augmented generation (RAG) code assistant using Ollama and LangChain to optimize your coding workflow.

December 30, 2024

by Santhosh Vijayabaskar

CORE

· 7,404 Views · 6 Likes

Setting Up a ScyllaDB Cluster on AWS Using Terraform

This article will teach you how to set up, deploy, manage, and scale ScyllaDB clusters on AWS using Terraform with this comprehensive step-by-step guide.

December 30, 2024

by Oleg Lipin

· 10,273 Views · 5 Likes

Bridging the Gap: Unlocking the Power of HDFS-Based Data Lakes With Streaming Databases

Big data technologies' quick development has highlighted the necessity of a smooth transition between real-time data analytics and batch processing systems.

December 30, 2024

by Gautam Goswami

CORE

· 3,113 Views · 1 Like

Getting Started With Snowflake Snowpark ML: A Step-by-Step Guide

This guide discusses setting up the Snowpark ML library, configuring your environment, and implementing a basic ML use case.

December 30, 2024

by Sevinthi Kali Sankar Nagarajan

· 4,121 Views · 2 Likes

Implementing CRUD Operations With NLP Using Microsoft.Extensions.AI

Learn how to implement CRUD operations using NLP in a .NET Web API application with the Microsoft.Extensions.AI library for a light management system.

December 30, 2024

by Baraneetharan Ramasamy

· 3,664 Views · 1 Like

AI in 2024: Year in Review and Predictions for 2025

The past year has been transformative for artificial intelligence. Let's review the major developments of 2024 and explore what the future might hold in 2025.

December 30, 2024

by Roger Oriol

· 2,645 Views

5 AI Trends That Will Define Software Development in 2025

In this article, we’ll explore the top AI trends that will shape software development in 2025: Agentic AI, Multimodal AI, Democratization of AI, and more.

December 27, 2024

by Sameer Danave

· 6,409 Views

Creating Your First GenAI RAG App: Sony TV Manual Example

For novices, this tutorial demonstrates how to chunk PDF manuals, embed them with OpenAI, retrieve relevant text via FAISS, and produce GPT-based answers using RAG.

December 27, 2024

by Shreyam Dutta Gupta

· 3,609 Views · 3 Likes

Advancing Enterprise AI Solutions With Agentic RAG

Agentic RAG revolutionizes enterprise AI solutions by combining RAG with autonomous AI agents, enabling smarter, more proactive, and task-oriented AI applications.

December 27, 2024

by Suri (thammuio)

CORE

· 26,098 Views · 3 Likes

Building Intelligent Multi-Agent Conversational Systems Using the AutoGen Framework

Learn how to build scalable multi-agent conversational AI systems easily using the AutoGen framework with modular and extensible features.

December 26, 2024

by Aravind Nuthalapati

CORE

· 23,424 Views · 3 Likes

Seamless RDS to DynamoDB Migration: Unlocking Scalability With the Dual Write Strategy

This article provides a detailed overview of the Incremental Migration with the Dual Write strategy, including the necessary steps, considerations, and best practices.

December 26, 2024

by Deepti Marrivada

· 7,264 Views · 2 Likes

AWS Performance Tuning: Why EC2 Autoscaling Isn’t a Silver Bullet

Deep dive into why Autoscaling isn’t a silver bullet and how engineers can augment it for better performance and cost efficiency.

December 25, 2024

by John Akkarakaran Jose

· 15,941 Views · 6 Likes

Bitmaps in Dragonfly: Compact Data With Powerful Analytics

Explore key commands, master bit-level operations, and dive into real-world use cases like user retention and feature flags.

December 25, 2024

by Joe Zhou

· 4,661 Views · 1 Like

Addressing the Problem of Drift Detection and Drift Cause Analysis

This article discusses a scenario where runtime environments deviate from their IaC-defined states, and drifts can have long-term implications.

December 25, 2024

by Omry Hay

· 6,189 Views · 3 Likes

How OpenAI’s Downtime Incident Teaches Us to Build More Resilient Systems

OpenAI's downtime incident reveals critical lessons in building resilient systems, improving testing, and preventing future failures in distributed environments.

December 25, 2024

by Vasanthi Govindaraj

· 9,743 Views · 2 Likes

The Future of Data Lies in Transformer Models vs. Big Data Transformations

Last year witnessed the explosive rise of large models, generating global enthusiasm and making AI seem like a solution to all problems.

December 25, 2024

by William Guo

· 5,353 Views · 1 Like

Creating Scalable, Compliant Cloud Data Pipelines in SaaS through AI Integration

Cloud-based AI data pipelines provide SaaS companies with scalable, cost-effective solutions for real-time insights and efficient data management.

December 24, 2024

by Venkata Gummadi

· 23,932 Views · 3 Likes

10+ Essential Visualization Libraries for Svelte Apps in 2024

An overview of top data visualization frameworks and libraries to help Svelte developers integrate charts, node-based diagrams, and project planning tools.

December 24, 2024

by Hanna Kowalska

· 5,858 Views

Exploring Binary Quantization in Vector Databases

Binary quantization in vector databases enhances efficiency, improves performance, and optimizes data storage for advanced information retrieval and analysis.

December 24, 2024

by Martin Summer

· 3,475 Views

The Latest Data Engineering Topics