Data Engineering Resources

Integration Architecture Guiding Principles, A Reference

The Integration Architecture guiding principles are guidelines to increase the consistency and quality of technology decision-making for the integration solutions.

April 24, 2023

by Susmit Dey

CORE

· 27,556 Views · 19 Likes

Get an In-Depth Look at the Best Ways To Store Application Parameters in AWS

Learn the top techniques for storing application parameters in AWS. Check out this comprehensive guide and optimize your cloud infrastructure today.

April 24, 2023

by Satrajit Basu

CORE

· 5,579 Views · 1 Like

Best Practices and Hacks: Data Loading in Snowflake From Stages

This second article of the series will help you to understand the best practices for file-based data loading in Snowflake.

April 24, 2023

by Kedhar Natekar

· 4,657 Views · 3 Likes

API Gateway Pattern: Features and the AWS Implementation

This article will provide a deep-dive study of API Gateways, starting with the problems that are addressed by API Gateways in enterprise computing

April 24, 2023

by Rodrigo Dinis

· 4,323 Views · 1 Like

10 Best Practices for Using Kubernetes Network Policies

In this article, we will explore ten best practices for using Kubernetes Network Policies to enhance the security and reliability of your applications.

April 24, 2023

by Emmanuel Moila

· 8,235 Views · 3 Likes

Make E2e Testing Easier With the Right Tools

E2e testing is notably slower and more expensive than lower-level tests. In this article, we will cover tools and approaches to make e2e testing easier.

April 24, 2023

by Mirza Sisic

· 3,118 Views · 1 Like

Generics in Java and Their Implementation

In Java programming language, generics are introduced in J2SE 5 for dealing with type-safe objects.

April 24, 2023

by Bikash Jain

· 13,039 Views · 10 Likes

Zero Trust Network for Microservices With Istio

Securing microservices networks in the cloud has become challenging. Istio service mesh with Zero Trust Network (ZTN) philosophy offers a solution.

April 23, 2023

by Anas T

· 8,471 Views · 10 Likes

API Gateway For ChatGPT Plugins

Learn how API Gateway can be beneficial for ChatGPT plugin developers to expose, secure, manage, and monitor their API endpoints.

April 23, 2023

by Bobur Umurzokov

· 4,983 Views · 2 Likes

Custom Training of Large Language Models (LLMs): A Detailed Guide With Code Samples

This article provides a comprehensive guide on how to custom-train large language models, such as GPT-4, with code samples and examples.

April 22, 2023

by Suresh Rajasekaran

· 25,175 Views · 4 Likes

How Midjourney and Other Diffusion Models Create Images from Random Noise

Diffusion models, such as Midjourney and DALL-E 2 can produce stunning images, but their training always begins with complete noise.

April 22, 2023

by Rostyslav Demush

· 8,251 Views · 2 Likes

MLOps for Beginners: Getting Started With MLOps

New to MLOps? Learn the basics of machine learning with this MLOps for beginners guide.

April 22, 2023

by Atulpriya Sharma

· 4,883 Views · 1 Like

Producing and Consuming Avro Messages With Redpanda Schema Registry

In this article, learn how to use Redpanda’s built-in schema registry to produce and consume Apache Avro messages in five simple steps.

April 21, 2023

by Dunith Dhanushka

CORE

· 4,998 Views · 1 Like

Lambda Architecture: A Powerful Approach to Data Engineering

In this article, we will explore the Lambda Architecture in detail, understanding its key concepts, benefits, and challenges.

April 21, 2023

by Amlan Patnaik

· 5,518 Views · 6 Likes

Why Use Kong API Gateway

Unlock the power of your API infrastructure with the Kong API Gateway. Let's explore the top reasons why you should choose Kong for API management needs.

April 21, 2023

by Vaibhav Pathak

· 7,822 Views · 4 Likes

How to Stream Sensor Data to Apache Pinot for Real Time Analysis

Learn how to use Arduino, MQTT, Kafka, and Apache Pinot to stream environmental data for real-time access to sensor data.

April 21, 2023

by David G. Simmons

CORE

· 7,152 Views · 8 Likes

Test Tool Tutorial: A Comprehensive Guide With Examples and Best Practices

This test tool tutorial will give you a detailed overview of the test tools, why you should use them, their benefits, and much more.

April 21, 2023

by Harshit Paul

· 6,594 Views · 1 Like

IoT Testing: How It Works and Why You Can’t Do Without It

This article explains IoT testing, how to do it correctly, which challenges you can face along the way, and why.

April 21, 2023

by Inna Martyniuk

· 4,158 Views · 2 Likes

Consumer Group in Kafka [Video Tutorials]: Partitions and Consumers

In this video tutorial series, take a deeper look at the consumer group in Kafka with varying numbers of partitions and consumers.

April 21, 2023

by Ram N

· 6,210 Views · 2 Likes

CockroachDB TIL: Volume 12

Learn to find partial indexes, identify the DB Console URL with SQL, experiment with a lightweight pooler called pgcat, handle client disconnects, and more.

April 20, 2023

by Artem Ervits

CORE

· 3,952 Views · 1 Like

The Latest Data Engineering Topics