Data Engineering Resources

Stream Processing in the Serverless World

Learn how real-time data processing enables businesses to gain actionable insights without the overhead of managing infrastructure.

September 30, 2024

by Rajesh Kumar Pandey

· 5,384 Views · 5 Likes

Using AI in Your IDE To Work With Open-Source Code Bases

In this article, learn how we can add enhancements to the langchaingo project with Amazon Q Developer support in VS Code.

September 28, 2024

by Abhishek Gupta

CORE

· 4,500 Views · 3 Likes

Leveraging IBM WatsonX Data With Milvus to Build an Intelligent Slack Bot for Knowledge Retrieval

This tutorial covers populating the database with IBM WatsonX Data or open-source Milvus and connecting the bot to Slack to answer queries in real-time.

September 27, 2024

by Pradeep Gopalgowda

· 5,748 Views · 4 Likes

Indexes Under the Hood

One of the most obvious improvements for databases is to use indexes. Let's see them in detail to understand how they work in PostgreSQL and what makes them useful.

September 27, 2024

by Adam Furmanek

CORE

· 5,462 Views · 9 Likes

Assigning Pods to Nodes Using Affinity Rules

Affinity and anti-affinity are Kubernetes features to help you manage the schedule of your applications. Explore two scenarios where to use these features.

September 27, 2024

by Rafael Natali

· 5,662 Views · 5 Likes

Is Spring AI Strong Enough for AI?

Explore Spring's capabilities within the AI domain, its potential integration with AI libraries, and its ability to effectively manage AI workflows.

September 27, 2024

by Reza Ganji

CORE

· 17,280 Views · 16 Likes

Low-Code AI Agent for Classifying User Support Tickets With OpenAI and Kumologica

This article discusses the design and implementation of an intelligent case classification scenario using an AI agent in Kumologica and the OpenAI platform.

September 27, 2024

by Pranav K

CORE

· 4,010 Views · 2 Likes

Advanced Strategies for Building Modern Data Pipelines

Modern data pipelines streamline the flow of data by making it faster and easier to collect and use them. Here is a better way to understand and create one.

September 26, 2024

by Fawaz Ghali, PhD

CORE

· 4,905 Views · 2 Likes

Integrating Data Engineering Into Artificial Intelligence

Importance of tight coupling between data engineering and artificial intelligence for data engineers and data scientists and best practices.

September 26, 2024

by Akshay Agarwal

· 3,990 Views · 4 Likes

Java for AI

Explore how the Java programming language can be used for AI development, along with supporting libraries and tools.

September 26, 2024

by SURESH SRIRANGAM

· 41,058 Views · 12 Likes

Angular RxJS Unleashed: Supercharge Your App With Reactive Operators

Master RxJS operators in Angular to efficiently manage data streams, handle errors, and enhance reactive programming in your apps.

September 26, 2024

by Raju Dandigam

· 5,864 Views · 4 Likes

Using Flink, Kafka, and NiFi for Real-Time Airport Arrivals and Departures

Learn to build a streaming application using the best of NiFi, Kafka, and Flink for event-driven apps. OpenSky networks rest feeds provide all the data.

September 26, 2024

by Timothy Spann

· 4,769 Views · 6 Likes

Backpressure in Distributed Systems

Learn more about backpressure, a technique in distributed systems to prevent overload and cascading failures by controlling the flow of requests.

September 26, 2024

by Rajesh Kumar Pandey

· 9,966 Views · 3 Likes

Building a Zero Trust API With ASP.NET Core: A Developer’s Guide

Secure your ASP.NET Core API with a Zero Trust model by authenticating every request, enforcing least privilege, and continuously monitoring to protect against threats.

September 25, 2024

by Aneesh Gopalakrishnan

· 6,614 Views · 4 Likes

Concurrency Issues in SQL and Distributed Systems

What happens if the first transaction must be canceled later for any reason? How should this affect the other transaction? Read on to understand!

September 25, 2024

by Adam Furmanek

CORE

· 5,665 Views · 6 Likes

Automating Data Pipelines With Snowflake: Leveraging DBT and Airflow Orchestration Frameworks for ETL/ELT Processes

This article explores the automation of data pipelines using Snowflake, dbt, and Airflow, detailing best practices for efficient data processing and orchestration.

September 25, 2024

by Harshavardhan Yedla

· 5,956 Views · 3 Likes

Feature Engineering Transforming Predictive Models

Delve into the transformative power of feature engineering in applied machine learning, and learn how carefully crafted features can elevate your models.

September 25, 2024

by Sundeep Goud Katta

· 4,476 Views · 4 Likes

Securing Your Azure Kubernetes Services Cluster

Learn about securing an Azure Kubernetes cluster with the principle of least privilege as a top priority integrated with Microsoft Entra and Kubernetes RBAC.

September 25, 2024

by Rafael Natali

· 5,205 Views · 2 Likes

Probabilistic Graphical Models: A Gentle Introduction

Explore this guide to probabilistic graphical models, which represent probability distributions and capture conditional independence structures using graphs.

September 25, 2024

by Salman Khan

CORE

· 31,403 Views · 49 Likes

The Future of Application Security: Empowering Developers in the AI Era

Explore the future of application security in the AI era. Learn how developers can integrate security into their workflow and leverage AI responsibly.

September 25, 2024

by Tom Smith

CORE

· 3,286 Views · 1 Like

The Latest Data Engineering Topics