Data Engineering Resources

Uber's Success With Apache Pinot: Cost Savings and Performance Boost

Uber migrated from Elasticsearch to Apache Pinot and saw a 70% reduction in costs, an 80% reduction in CPU cores, and a decrease in page load times and query timeouts.

April 29, 2024

by Kristin Crosier

· 1,849 Views · 3 Likes

How To Create Auto Scaling in AWS

This blog post will help you create Auto Scaling in AWS, a collection of Amazon Ec2 instances with similar characteristics among them in just a group.

April 29, 2024

by Alfonso Valdes

· 1,972 Views · 1 Like

Gain Lightning Fast Reads and Writes Without Changing Your PostgreSQL Application

Walk through how the YugabyteDB distributed database lowers read and write latency and (most importantly) runs an existing PostgreSQL application with no costly rewrite.

April 29, 2024

by Jim Knicely

· 1,178 Views · 1 Like

Multiplatform Directory Bookmarks on the Command Line

This article discusses how to manage directory bookmarks on the command line for efficient use of the character terminal.

April 29, 2024

by Pavel Ponec

· 1,769 Views · 1 Like

Adjusting Data Before Sending It to Kentik NMS

Sometimes, you want to modify data before displaying it in your network monitoring system. So strap in and let’s get mathy!

April 29, 2024

by Leon Adato

· 1,485 Views · 1 Like

Why Cloud Native Is Vital to Your Organization's APIs: The Impact Could Be More Than Expected

As your API infrastructure expands, a cloud-native design provides the necessary tooling to ease supportability and manageability efforts.

April 29, 2024

by John Vester

CORE

· 59,049 Views · 2 Likes

The Simplest Ingredient for Developing Real-Time Front-End Applications

How to build a real-time frontend application using Redpanda Serverless, Pusher, and Vercel. Managing streaming data, and setting up real-time communication channels.

April 29, 2024

by Christina Lin

CORE

· 4,725 Views · 4 Likes

How To Use Retrieval Augmented Generation (RAG) for Go Applications

In this article, learn how to implement RAG (using LangChain and PostgreSQL) to improve the accuracy and relevance of LLM outputs.

April 29, 2024

by Abhishek Gupta

CORE

· 3,350 Views · 2 Likes

New All-Flash Solutions To Streamline Data Management for AI and High-Volume Workloads

Quantum's all-flash solutions simplify data management for AI and high-volume workloads, offering high-performance storage and seamless integration.

April 29, 2024

by Tom Smith

CORE

· 1,940 Views · 1 Like

Turbocharge Innovation With Automated API Generation: Leveraging Automated API Generation for a Competitive Advantage

Amplify the potential of generative AI and low- and no-code platforms in API generation by highlighting their roles in fostering innovation and expediting time to market.

April 28, 2024

by Pratik Prakash

CORE

· 3,932 Views · 4 Likes

Get Some Rest! A Full API Stack

Explore options for building an API tech stack — from initial API design and documentation to API development, lifecycle management, security, monitoring, and more.

April 27, 2024

by Thomas Jardinet

CORE

· 4,783 Views · 7 Likes

Revisiting Observability: A Deep Dive Into the State of Monitoring, Costs, and Data Ownership in Today’s Market

Explore insights into the current challenges in monitoring, highlighting issues with instrumentation, pricing models, and data ownership.

April 26, 2024

by Mike Moore

· 2,788 Views · 3 Likes

Kubernetes: Future or Fad?

I don't think Kubernetes is yet another fad, but I strongly hope it's not going to be our future either. At least not in its current incarnation.

April 26, 2024

by Christoph Engelbert

· 2,729 Views · 2 Likes

Solving PostgreSQL Indexes, Partitioning, and LockManager Limitations

Read this blog to learn how to avoid the scalability limitations of PostgreSQL's partition-based approach by leveraging automatic sharding and global indexing.

April 26, 2024

by Franck Pachot

· 1,649 Views · 1 Like

Auto-Increment Columns in Apache Doris

Auto-increment columns in Apache Doris accelerate dictionary encoding and pagination without damaging data writing performance.

April 26, 2024

by Shirley H.

· 1,703 Views · 1 Like

Diffusion and Denoising: Explaining Text-To-Image Generative AI

Denoising diffusion models are trained to pull patterns out of noise, to generate a desirable image. This article illustrates the different ways to use diffusion.

April 26, 2024

by Kevin Vu

· 1,520 Views · 1 Like

Understanding API Technologies: A Comparative Analysis of REST, GraphQL, and Asynchronous APIs

Learn about the advantages vs. disadvantages and ideal use cases for REST, GraphQL, and Async APIs in this concise guide.

Updated April 26, 2024

by Saurabh Dashora

CORE

· 7,653 Views · 7 Likes

The Case for Choosing Between TCP and UDP for Data Transmission

The article compares TCP and UDP for data transmission, weighing reliability and sequential delivery of packets or fast speed and packet delivery in a stream.

April 26, 2024

by John Ejiofor

· 2,968 Views · 1 Like

Revolutionizing Business Intelligence: The Role of Explainable AI and Empowering Non-Technical Users

Explore the future of Business Intelligence (BI) with insights on explainable AI, empowering non-technical users, and the evolving role of data teams.

April 26, 2024

by Tom Smith

CORE

· 1,836 Views · 1 Like

Enhancing Web Scraping With Large Language Models: A Modern Approach

Explore the integration of Large Language Models (LLMs) with web scraping and the use of LLMs to efficiently transform complex HTML into structured JSON.

April 25, 2024

by Nacho Corcuera

· 7,243 Views · 3 Likes

The Latest Data Engineering Topics