Data Engineering Resources

Breaking Down Data Silos With a Unified Data Warehouse: An Apache Doris-Based CDP

An insurance company uses Apache Doris in replacement of Spark+Impala+HBase+NebulaGraph, in their Customer Data Platform for 4 times faster customer grouping.

March 19, 2024

by Shirley H.

· 7,463 Views · 2 Likes

Using My New Raspberry Pi To Run an Existing GitHub Action

Experiment with self-hosted runner Raspberry Pi, using it to run an existing GitHub action. Read more about the author's findings!

March 18, 2024

by Nicolas Fränkel

CORE

· 7,011 Views · 2 Likes

Rethinking DevOps in 2024: Adapting to a New Era of Technology

Emerging technologies, evolving methodologies, and changing business needs are redefining what it means to implement DevOps practices effectively.

March 18, 2024

by Ankush Madaan

· 4,613 Views · 1 Like

LangChain, Python, and Heroku

In this tutorial, learn how to build and deploy LLM-based applications with ease using LangChain, Python, and Heroku for streamlined development and deployment.

March 18, 2024

by Alvin Lee

CORE

· 3,904 Views · 3 Likes

Power BI: Transforming Banking Data

As a data architect or data engineer, you know how vital it is to fully understand the power of your data. This task holds even more gravity in the banking sector.

March 18, 2024

by Serter Solak

· 35,609 Views · 7 Likes

Integrating Snowflake With Trino

Trino simplifies accessing Snowflake, highlighting its versatility in providing robust data management and analytics solutions for modern enterprises.

March 18, 2024

by Artem Ervits

CORE

· 13,436 Views · 1 Like

Crafting a Custom Sports Activity Service With OpenAI and Node.js

Development journey of a Node.js application with OpenAI integration, craft a tailored sports activity service that dynamically adapts to unique fitness indicators.

March 18, 2024

by Anton Kalik

· 6,508 Views · 1 Like

AI-Driven API and Microservice Architecture Design for Cloud

Incorporating AI into API and microservice architecture design for the Cloud can bring numerous benefits. Here are some key aspects.

March 18, 2024

by Nilesh Charankar

· 13,794 Views · 2 Likes

Securing AWS RDS SQL Server for Retail: Comprehensive Strategies and Implementation Guide

This article provides a detailed guide on securing AWS RDS SQL Server instances, tailored for retail stores, with practical setup examples.

March 18, 2024

by Vijay Panwar

CORE

· 3,375 Views · 1 Like

Building a Sustainable Data Ecosystem

Generative AI brings innovation, but it also brings challenges in privacy, fairness, and accountability in data sharing.

March 15, 2024

by Chidumga Izuzu

· 5,814 Views · 6 Likes

Understanding Softmax Activation Function for AI/ML Engineers

The article explains the softmax function's role in turning neural network outputs into probabilities for classification in machine learning.

March 15, 2024

by Pavan Belagatti

CORE

· 3,073 Views · 1 Like

Understanding the Power of AWS Organizations: Streamlining Cloud Management

AWS Organizations is a cloud management service provided by Amazon Web Services (AWS) that simplifies the administration of multiple AWS accounts within an organization.

March 15, 2024

by Saikrishna Chinthapatla

· 3,820 Views · 1 Like

Optimizing Model Training: Strategies and Challenges in Artificial Intelligence

When you train a model, you send data through the network multiple times. Think of it like wanting to become the best basketball player.

March 15, 2024

by Mirza Naeem

· 3,388 Views · 1 Like

Azure Stream Analytics Upsert Operation Using Cosmos DB

Upsert operations allow you to update existing records or insert new ones based on specific criteria within your streaming pipelines.

March 15, 2024

by Amlan Patnaik

· 4,162 Views · 1 Like

How to Setup RAID10 on Dell Servers

Understanding different types of storage configurations is important and today we will discuss how we can achieve redundancy and performance by using RAID arrays.

March 15, 2024

by Prashanth Ravula

CORE

· 6,169 Views · 5 Likes

Data Flow Diagrams for Software Engineering

This article provides fundamental knowledge about DFDs, highlighting their benefits and guiding you on how to leverage them effectively.

March 14, 2024

by Stelios Manioudakis, PhD

CORE

· 15,105 Views · 5 Likes

Data Lake vs. Data Warehouse: 10 Key Differences

In this article, learn more about the ten major differences between data lakes and data warehouses to make the best choice.

March 14, 2024

by Waris Husain

· 3,572 Views · 1 Like

Building Safe AI: A Comprehensive Guide to Bias Mitigation, Inclusive Datasets, and Ethical Considerations

Data quality is key for fair AI. Biased or incomplete datasets lead to AI models that make unfair or inaccurate decisions, harming individuals and eroding trust.

March 14, 2024

by Ramakrishnan Neelakandan

· 4,699 Views · 3 Likes

Mastering AWS API Gateway: A Comprehensive Guide To Setting up Routes, Integrations, and Deployments

In this article, we will walk through the process of setting up AWS API Gateway with a Network Load Balancer (NLB), Target Group, Security Group, and VPC.

March 14, 2024

by Vijay Panwar

CORE

· 4,930 Views · 1 Like

Qumulo Unifies Data Across the Enterprise With Scale Anywhere Data Platform

Qumulo's CTO explains how the company's Scale Anywhere Data Platform unifies unstructured data across edge, core, and cloud for easy management and access.

March 14, 2024

by Tom Smith

CORE

· 2,681 Views · 1 Like

The Latest Data Engineering Topics