Data Engineering Resources

The 3 Stages of an Effective Test Data Strategy

Testing is crucial for delivering quality applications quickly, but providing the right test data remains a major bottleneck.

October 17, 2023

by Rich Jordan

· 4,005 Views · 2 Likes

Kafka Event Streaming AI and Automation

Explore how to use ChatGPT to create an IoT Kafka event consumer and API Logic Server to logic to produce temperature reading events outside a defined range.

October 17, 2023

by Tyler Band

· 29,404 Views · 8 Likes

Enhanced Security for Your Secrets With AWS Secrets Manager

AWS Secrets Manager secures credentials with encryption and access controls while providing robust protection against threats.

October 17, 2023

by Raghava Dittakavi

CORE

· 5,285 Views · 3 Likes

Future Trends in Data Integration

Exploring the future of data integration, from cloud solutions and real-time analytics to machine learning. Adaptability is key in this evolving landscape.

October 17, 2023

by Jeffrey Faber

· 3,671 Views · 3 Likes

SQL Data Manipulation Language (DML) Operations: Insert, Update, Delete

This article covers SQL DML operations like insert, update, delete, and more, including drop, distinct, and group by.

October 17, 2023

by Madhuri Hammad

· 18,240 Views · 5 Likes

The Impact of IoT on Data Integration: A Transformational Journey

IoT's impact on data integration is profound, driving shifts in storage methods, real-time processing, and architectural paradigms.

October 17, 2023

by Ralph Burgess

· 3,391 Views · 1 Like

Fine Tuning LLM: Parameter Efficient Fine Tuning (PEFT), LoRA and QLoRA: Part 1

In this blog, we will understand the idea behind Parameter Efficient Fine Tuning (PEFT) and explore LoRA and QLoRA, Two of the most important PEFT methods.

October 16, 2023

by A B Vijay Kumar

· 5,525 Views · 2 Likes

Oracle Outlines Vision for AI and Cloud-Driven Future

Larry Ellison believes generative AI is transformational and details the unique advantages of Oracle Cloud for building enterprise AI apps

October 16, 2023

by Tom Smith

CORE

· 5,018 Views · 2 Likes

A Deep Dive Into the Call Chain Relationship Between Presto, Hive, and Alluxio

Understanding how Presto+Hive+Alluxio work together and the flow from SQL query to low-level file system operations is key to tuning performance.

October 16, 2023

by Jiaming Mai

· 2,932 Views · 2 Likes

How Is Apache APISIX Fast?

Apache APISIX achieves high performance by using etcd for storage, hash tables for storing IP addresses, and radix trees for routing.

October 16, 2023

by Navendu Pottekkat

· 3,655 Views · 2 Likes

AWS Lambda Pricing for a Serverless Application

In this blog post, we’ll briefly compare AWS Lambda Pricing with other Cloud providers and discuss the key elements of the AWS Lambda pricing model.

October 16, 2023

by Rahul Shivalkar

· 4,201 Views · 1 Like

Nebraska.Code() — Developing in the Great Plains

With topics ranging from OSS contribution to getting along with your coworkers to using Azure Key Vault at scale, there was a lot of knowledge shared.

October 16, 2023

by Dwayne McDaniel

· 4,039 Views · 2 Likes

Unlocking Efficient AuthZ With Cerbos’ Query Plan

Cerbos efficiently handles the majority of access control decisions for standard application requirements. Its query plan handles the most challenging cases.

October 16, 2023

by Aldin Kiselica

· 2,626 Views · 2 Likes

The Future of AI in Software Development: Trends and Innovations

Explore the ever-changing landscape of AI in software development. Discover emerging trends and innovations shaping the future of coding and programming.

October 16, 2023

by Megha Verma

· 4,566 Views · 2 Likes

Autoscaling Kubernetes Worker Nodes With Karpenter

How to set up and use Karpenter for autoscaling and collecting underutilized spot and on-demand instances for AWS EKS.

October 16, 2023

by Anadi Misra

· 4,607 Views · 2 Likes

Understanding Supervisor in Apache Druid

A supervisor is a built-in part of Druid, making it easier to ingest, analyze, and monitor data in real-time. Learn more!

October 16, 2023

by Gautam Goswami

CORE

· 3,201 Views · 3 Likes

Best Practices for Picking PostgreSQL Data Types

In this article, learn which data types best suit your application when storing massive data volumes in PostgreSQL and Timescale.

October 16, 2023

by kelvin steve

· 5,085 Views · 1 Like

Unlocking the Cloud's Best-Kept Secret: Maximize Your Website's Potential With AWS S3 Static Hosting

The article highlights the advantages of hosting static websites on S3, such as high availability, scalability, unlimited bandwidth, and robust security features.

October 16, 2023

by Satrajit Basu

CORE

· 3,863 Views · 1 Like

Platform Engineering Trends in Cloud-Native: Q&A With Tom Wilkie

How Kubernetes has lowered marginal costs of services, enabled composability, and driven an explosion of telemetry data needing improved observability.

October 16, 2023

by Tom Smith

CORE

· 4,443 Views · 3 Likes

How Can Software Developers Be Useful With ChatGPT and Bard AI?

Previously, developers used to spend a lot of time in code or debugging, but now, with the help of ChatGPT or Bard AI, they transform the software engineers easily and efficiently.

October 16, 2023

by Simran Sumaiya

· 3,220 Views · 1 Like

The Latest Data Engineering Topics