Data Engineering Resources

How Enabling Slow Query Log Enhances Postgres Observability

Here, learn about the slow query log in PostgreSQL, a feature that allows you to log queries that take longer than a specified threshold to execute.

May 2, 2024

by Adam Furmanek

CORE

· 1,774 Views · 1 Like

Preventing Account Takeover Through Open and Centralized Compromised Credential Index

Account Takeover frauds cause billions of dollars to people and companies. An open security standard could prevent this across software applications.

May 2, 2024

by Chetan Honnenahalli

· 1,284 Views · 2 Likes

Optimizing Azure DevOps Pipelines With AI and Continuous Integration

This article helps in leveraging AI to help detect issues, optimize workflows, and deliver high-quality software efficiently.

May 2, 2024

by Naga Santhosh Reddy Vootukuri

CORE

· 4,856 Views · 3 Likes

Dataframes With Big Data in Couchbase

In this article, we'll explore how users more comfortable with dataframes can work with their Big Data in Couchbase's Analytics Service.

May 2, 2024

by Glenn Galvizo

· 1,242 Views · 1 Like

How To Fix Kafka Consumers Due To Corrupted Log Files or Missing Log Files

This article explores recovering Kafka Brokers from corrupted data storage and recovering Kafka consumers due to missing log files.

May 2, 2024

by Ramu kakarla

· 2,237 Views · 1 Like

Understanding Gen AI: RAG Implementation

RAG is a technique to use existing LLMs to generate contextual responses with proprietary data, without the need to retrain the LLMs, offering a cost-effective solution.

May 2, 2024

by Ganesh Kumar Murugesan

· 1,985 Views · 2 Likes

Five Ways To Pass Parameters to Apache APISIX

This article explores all the ways to send parameters to Apache APISIX, which is beneficial when you write a custom plugin.

May 2, 2024

by Nicolas Fränkel

CORE

· 1,509 Views · 1 Like

Navigating Cloud-Native Components: Using Service Registry and Definitions

Explore the service registry and service definitions, which are crucial components in cloud-native architecture for efficient, reliable, and scalable applications.

May 2, 2024

by Gaurav Gaur

CORE

· 1,576 Views · 3 Likes

Exploring FAPI 2.0: The Latest in API Security Technology

This post explores FAPI standards, which enhance secure data exchange in high-risk sectors, evolving from OAuth 2.0-based FAPI 1.0 to the more secure FAPI 2.0.

May 1, 2024

by Hideki Ikeda

· 1,841 Views · 2 Likes

Data Warehouse for Data Science: Adopting Arrow Flight SQL for 10X Data Transfer

Apache Doris 2.1 supports Arrow Flight SQL protocol for reading data from Doris. It delivers tenfold speedups compared to PyMySQL and Pandas.

May 1, 2024

by Shirley H.

· 1,172 Views · 2 Likes

Reliability Models and Metrics for Test Engineering

Explore how reliability models and metrics help identify potential weak spots, anticipate failures, and build better products.

May 1, 2024

by Stelios Manioudakis, PhD

CORE

· 5,227 Views · 2 Likes

An Introduction to Stream Processing

As a gentle introduction to stream processing, we’ve defined what it is, how it differs from batch processing, and how computation works in stream processing.

May 1, 2024

by Federico Trotta

· 2,282 Views · 4 Likes

API-Based Web Service as Serverless Function

Serverless architecture is a method of decoupling server maintenance, scalability, and availability from the development process.

May 1, 2024

by Madhan Muralimanohar

· 1,263 Views · 1 Like

Files and Exceptions in Python

Exceptions are special objects that Python uses to manage errors that occur when a program is running. Here, learn to work with files and handle exceptions.

April 30, 2024

by Duncan Ndegwa

· 1,035 Views · 2 Likes

The Evolution of Platform Extensibility

From API to AI, this article attempts to capture the state of extensibility and charts a course for what the future might hold.

April 30, 2024

by Ashwin Ganesh Krishnamurthy

· 1,052 Views · 1 Like

Exploration of Azure OpenAI

Follow an overview of Azure Open AI and its significance in the field of artificial intelligence, learning how to build intelligent applications.

April 30, 2024

by Naga Santhosh Reddy Vootukuri

CORE

· 1,217 Views · 3 Likes

Expose Your Database as a REST API Safely and Without Code

DB2Rest is open-source middleware that offers a no-code way to safely expose data from your database for your applications to consume.

April 30, 2024

by Thad Guidry

· 1,513 Views · 2 Likes

Designing Communication Architectures With Microservices

Communication between microservices should be robust and efficient.

April 30, 2024

by Swathi Prasad

· 3,169 Views · 3 Likes

AI-Driven Development: Maximizing Efficiency With GitHub Copilot and Seamless Chat Collaboration in Visual Studio IDE

GitHub Copilot AI assistant speeds up development by providing automatic code completions, suggestions, and many more that enhance developer's productivity.

April 30, 2024

by Naga Santhosh Reddy Vootukuri

CORE

· 1,490 Views · 3 Likes

AI Is BS Spelled AI

What is AI? In this opinion piece, follow along with the discussion of whether artificial intelligence or virtual assistance is the end goal.

April 29, 2024

by Greg Hall

· 695 Views · 1 Like

The Latest Data Engineering Topics