Data Engineering Resources

Auto-increment columns in Apache Doris accelerate dictionary encoding and pagination without damaging data writing performance.

April 26, 2024

by Shirley H.

· 1,839 Views · 1 Like

Diffusion and Denoising: Explaining Text-To-Image Generative AI

Denoising diffusion models are trained to pull patterns out of noise, to generate a desirable image. This article illustrates the different ways to use diffusion.

April 26, 2024

by Kevin Vu

· 1,569 Views · 1 Like

Understanding API Technologies: A Comparative Analysis of REST, GraphQL, and Asynchronous APIs

Learn about the advantages vs. disadvantages and ideal use cases for REST, GraphQL, and Async APIs in this concise guide.

Updated April 26, 2024

by Saurabh Dashora

CORE

· 7,967 Views · 7 Likes

The Case for Choosing Between TCP and UDP for Data Transmission

The article compares TCP and UDP for data transmission, weighing reliability and sequential delivery of packets or fast speed and packet delivery in a stream.

April 26, 2024

by John Ejiofor

· 3,162 Views · 1 Like

Revolutionizing Business Intelligence: The Role of Explainable AI and Empowering Non-Technical Users

Explore the future of Business Intelligence (BI) with insights on explainable AI, empowering non-technical users, and the evolving role of data teams.

April 26, 2024

by Tom Smith

CORE

· 1,907 Views · 1 Like

Enhancing Web Scraping With Large Language Models: A Modern Approach

Explore the integration of Large Language Models (LLMs) with web scraping and the use of LLMs to efficiently transform complex HTML into structured JSON.

April 25, 2024

by Nacho Corcuera

· 7,452 Views · 3 Likes

Handling Vectors in AI Context via PostgreSQL pgVector

An introduction to PostgreSQL pgVector extension and its supported metrics — L2 squared distance (Euclidian), dot product, cosine distance/similarity.

April 25, 2024

by Horatiu Dan

CORE

· 1,833 Views · 5 Likes

VMware vSphere Backup Methods: Ensuring Data Safety and Recovery

Explore various backup methods available for VMware vSphere environments, how they function, and their respective advantages and limitations.

April 25, 2024

by Aditya Bhuyan

· 2,096 Views · 2 Likes

Monetizing Generative AI Starts With Metering

Establishing clear usage and utility telemetry from AI features justifies substantial development costs and cloud infrastructure investments.

April 25, 2024

by Jason English

· 1,469 Views · 1 Like

A Complete Guide to the Real-Time Streaming Protocol (RTSP)

RTSP enables control of media streaming sessions for IoT video applications like surveillance cameras. Here's everything you need to know.

April 25, 2024

by Carsten Rhod Gregersen

· 3,080 Views · 4 Likes

Leveraging All-Flash Object Storage for Accelerating AI and Massive Data Workloads

All-flash object storage accelerates AI pipelines and data lakes, delivering performance, scalability, and cost efficiency for massive unstructured data workloads.

April 25, 2024

by Tom Smith

CORE

· 2,056 Views · 1 Like

Java Container Application Memory Analysis

In this article, I cover types of memory usage for a Java container application and methods to analyze memory consumption.

April 24, 2024

by Parveen Saini

· 2,991 Views · 2 Likes

Understanding LLMs: Mixture of Experts

Mixture of Experts is a LLM architecture that powers your most popular LLMs. Continue reading to learn about how they work and why they work so well.

April 24, 2024

by Roger Oriol

· 1,380 Views · 1 Like

How To Integrate Weather Tracking Into Web Applications With PHP

To track weather using PHP, weather data from a reliable source is required. Several weather APIs are available that provide real-time and forecast weather information.

April 24, 2024

by Santosh Sahu

· 1,506 Views · 2 Likes

How To Build Generative AI Apps on AWS Using Anthropic Claude 3

Explore building generative AI applications using Anthropic Claude 3 on AWS. Discover cloud-based AI innovation and advanced techniques in our guide.

April 24, 2024

by Nisar Ahmad

· 2,833 Views · 3 Likes

HTTP API: Key Skills for Smooth Integration and Operation (Part 1)

In this article, I want to share my experience on how to work properly with HTTP APIs of third-party or internal services

April 24, 2024

by Filipp Shcherbanich

CORE

· 2,452 Views · 4 Likes

Debugging Terraform Like a Pro: Strategies for Efficient Infrastructure Management

Mastering debugging in Terraform by identifying and resolving issues in Terraform configurations, states, and workflows.

April 24, 2024

by Josephine Eskaline Joyce

CORE

· 3,647 Views · 4 Likes

Starburst Unveils Fully Managed 'Icehouse' for Near Real-Time Analytics on the Open Data Lakehouse

Icehouse on Galaxy simplifies data analytics, reduces costs, and enables AI/ML workloads with an open data lakehouse architecture.

April 24, 2024

by Tom Smith

CORE

· 2,367 Views · 2 Likes

A Comparative Exploration of LLM and RAG Technologies: Shaping the Future of AI

Follow a comparative journey between LLM and RAG, shedding light on their mechanisms, applications, and the unique advantages they offer to the AI field.

April 24, 2024

by Ashok Gorantla

CORE

· 22,367 Views · 6 Likes

Why Do Data Teams Need To Adopt “Product Thinking”?

Explore the evolving landscape of Product Management in data teams and how adopting a product mindset is key for Data Product success.

April 23, 2024

by Yuliia Tkachova

· 579 Views · 1 Like

The Latest Data Engineering Topics