Data Engineering Resources

Mastering Back-End Design Patterns for Scalable and Maintainable Systems

Learn how back-end design patterns can simplify development, enhance scalability, and make your codebase cleaner, testable, and easier to maintain.

December 17, 2024

by Gautam Solaimalai

· 2,268 Views · 5 Likes

Configuring Autoscaling for Various Machine Learning Model Types

This tutorial helps explore options for choosing the right autoscaling policy for maximum resource utilization using AWS Sagemaker.

December 17, 2024

by Koushik Balaji Venkatesan

· 5,013 Views · 6 Likes

Building a New Public API

How I built a public API from scratch: A solo developer's journey through tech choices, security challenges, and creating real value. Learn key API project strategies.

December 16, 2024

by Ollie Bannister

· 4,014 Views · 1 Like

Mainframe to Serverless Migration on AWS: Challenges and Solutions

This article provides references to resources on mainframe-to-AWS cloud migration, serverless architecture, data migration, performance, and best practices.

December 16, 2024

by Sajith Narayanan

· 10,649 Views · 3 Likes

Integrating Llama 3.2 AI With Amazon SageMaker

Implement and deploy Llama 3.2 using Amazon SageMaker for generative AI tasks like content creation, conversational agents, and personalized recommendations.

December 16, 2024

by Sunil Sharma

· 4,344 Views · 1 Like

Dropwizard vs. Micronaut: Unpacking the Best Framework for Microservices

Discover the best option for building efficient and scalable microservices by reading this comparison of Dropwizard and Micronaut frameworks.

December 16, 2024

by Nilesh Jain

· 10,491 Views · 5 Likes

Securing APIs in Modern Web Applications

Learn about key practices for securing APIs in modern web apps, including authentication, encryption, and monitoring, to guard against vulnerabilities.

December 16, 2024

by Michal Clarke

· 3,195 Views · 2 Likes

Launching Pega Web Mashup Forms on a Secure Static Website With AWS S3

Learn how to configure and deploy Pega Web Mashup forms on a static website hosted in an AWS S3 bucket for seamless user access and simplified infrastructure.

December 16, 2024

by Sridevi Kakolu

· 2,272 Views · 3 Likes

Which Flow Is Best for Your Data Needs: Time Series vs. Streaming Databases

While both are used to handle time-related data, their underlying technologies and main purpose are built to serve different purposes.

December 13, 2024

by Gautam Goswami

CORE

· 4,978 Views · 3 Likes

Improve Your Agile Processes With Artificial Intelligence

AI enhances Agile by automating tasks, improving decisions, and optimizing workflows. Boost efficiency and deliver faster with AI-powered Agile.

December 13, 2024

by Liza Kosh

· 3,972 Views · 3 Likes

How to Fix SQL Database Stuck in Recovery Mode

Learn how to resolve an SQL database stuck in recovery mode. Discover phases of SQL database recovery, reasons that may cause the issue, and ways to bring it back to a consistent state.

Updated December 12, 2024

by Daniel Calbimonte

· 194,259 Views · 3 Likes

Databricks 101: An Introductory Guide on Navigating and Optimizing This Data Powerhouse

Gain insight into Databrick’s workspace, the pivotal role of workflows, and the nuanced world of compute resources.

December 12, 2024

by Noa Shavit

· 4,906 Views · 1 Like

Idempotency and Reliability in Event-Driven Systems: A Practical Guide

In this guide, learn how implementing robust idempotent patterns and leveraging tools can mitigate the risks posed by retries and duplicate events.

December 12, 2024

by Vikram Mohanagandhi

· 6,475 Views · 2 Likes

DevOps Tutorial: Docker, Kubernetes, and Azure DevOps

In this article, we break down everything you need to know about DevOps, so that you can get started building your own CI/CD pipeline.

Updated December 11, 2024

by Ranga Karanam

CORE

· 42,871 Views · 18 Likes

How and Why the Developer-First Approach Is Changing the Observability Landscape

Developers play a crucial role in modern companies. Explore more about the need to have a developer-first approach and include observability from day one.

December 11, 2024

by Adam Furmanek

CORE

· 5,689 Views · 3 Likes

How to Test PUT Requests for API Testing With Playwright Java

This tutorial demonstrates how to test PUT requests using the Playwright Java framework for API testing, including examples of updating data of an API.

December 11, 2024

by Faisal Khatri

CORE

· 9,715 Views · 2 Likes

Data Processing With Python: Choosing Between MPI and Spark

Know the differences between MPI and Spark for big data processing and find out which framework suits your needs for parallel and distributed computing.

December 11, 2024

by Anil Kumar Moka

· 4,999 Views

DuckDB Optimization: A Developer's Guide to Better Performance

DuckDB is a powerful in-memory database that offers parallel processing capabilities and this guide will outline optimization techniques to get the most out of it.

December 11, 2024

by Anil Kumar Moka

· 5,416 Views · 4 Likes

Setting Up DBT and Snowpark for Machine Learning Pipelines

This guide describes how to install and set up DBT and Snowpark for machine-learning model pipelines to streamline data processing and feature engineering tasks.

December 11, 2024

by Sevinthi Kali Sankar Nagarajan

· 7,756 Views · 3 Likes

Duration Bias in Video Recommendations: A Complete Guide to Fair Content Ranking

Explore the theoretical foundations and practical strategies for addressing duration bias to create balanced, fair, and effective recommendation systems.

December 11, 2024

by Amey Porobo Dharwadker

· 2,962 Views · 2 Likes

The Latest Data Engineering Topics