Data Engineering Resources

Introduction To Backtesting Strategy: Historical Data Replay in DolphinDB

Import historical data into a stream table in chronological order as “real-time data” so that the same script can be used both for backtesting and real-time trading.

March 1, 2024

by Davis Zhou

· 3,641 Views · 1 Like

Securing and Monitoring Your Data Pipeline: Best Practices for Kafka, AWS RDS, Lambda, and API Gateway Integration

Learn how to implement a data pipeline that integrates Apache Kafka with AWS RDS and use AWS Lambda and API Gateway to feed data into a web application.

February 29, 2024

by Vijay Panwar

CORE

· 6,722 Views · 1 Like

The article introduces the AWS SNS Fanout architecture as a solution for high-speed message distribution and highlights its scalability, reliability, and cost-efficiency.

February 29, 2024

by Satrajit Basu

CORE

· 6,471 Views · 3 Likes

The Evolution of Database Architectures: Navigating Big Data, Cloud, and AI Integration

The article discusses how database architectures have evolved to handle Big Data, Cloud, and AI, covering RDBMS, NoSQL, and cloud-based solutions.

February 29, 2024

by Pankaj Zanke

· 6,986 Views · 4 Likes

Real-Time Communication Protocols: A Developer's Guide With JavaScript

Explore WebSocket for low latency communication, WebRTC for peer-to-peer streaming, and MQTT for IoT while implementing provided JavaScript examples.

February 29, 2024

by Arun Pandey

CORE

· 12,879 Views · 6 Likes

Logging and Monitoring in Microsoft Azure

In this article, we will explore the importance of logging and monitoring in Azure, as well as various options and best practices for logging and monitoring.

February 29, 2024

by Aditya Bhuyan

· 6,136 Views · 1 Like

Swift Predicate: Usage, Composition, and Considerations

Predicate allows developers to filter and evaluate data collections. This article aims to explore the usage, structure, and key considerations of Swift Predicate.

February 29, 2024

by Fatbobman Xu

· 3,215 Views · 1 Like

Machine Learning at the Edge: Enabling AI on IoT Devices

Implementing ML on IoT devices demands efficient software engineering. Optimize models, use hardware accelerators, manage resources, and prioritize security for success.

February 29, 2024

by Deep Manishkumar Dave

· 4,372 Views · 1 Like

Exploring Four Unique Key Generation Strategies in GridGain and Apache Ignite

Creating "autoincrement" fields in a distributed system such as Apache Ignite is difficult. In this piece, we explore four different options and their tradeoffs.

February 29, 2024

by Stephen Darlington

· 2,798 Views · 1 Like

Deploy Your Own AI Chat Buddy

This comprehensive guide introduces the Qwen Chat Model, a powerful Generative AI and Large Language Model (LLM) capable of performing various tasks.

February 29, 2024

by Farruh Kushnazarov

· 2,878 Views · 1 Like

Understanding Multi-Tenancy: Core Logic and High-Level Code With Django

Custom implementations of multi-tenancy can allow developers to innovate and tailor solutions according to unique business needs and use cases.

February 29, 2024

by Jatinkumar Patel

· 3,420 Views · 1 Like

Semantic Search With Weaviate Vector Database

Learn how to implement a vector similarity search using Weaviate, a vector database.

February 28, 2024

by Gunter Rotsaert

CORE

· 3,206 Views · 1 Like

What Is Deep Learning and How Does It Power Modern AI?

Taking a close look at deep learning technologies, such as neural networks, to understand the role they play in creating modern AI.

February 28, 2024

by Ian Deed

· 4,941 Views · 3 Likes

Simplifying SQL Server and Kubernetes Across Hybrid/Multi-Cloud

Enabling automated configuration and streamlined management and deployment across on-premise, public cloud, and hybrid infrastructure.

February 28, 2024

by Tom Smith

CORE

· 6,012 Views · 1 Like

Advancements and Capabilities in Modern Mainframe Architecture

These advancements in mainframe architecture contribute to the platform's reliability, scalability, performance, and flexibility.

February 28, 2024

by Ravikiran Kandepu

· 7,018 Views · 1 Like

Model as a Service in the Generative AI Era

Learn about a cloud-based service where machine learning or Generative AI models are hosted in the cloud and are easily available for consumption through chat-based APIs.

February 28, 2024

by Bhala Ranganathan

CORE

· 5,184 Views · 2 Likes

The 7 Rs of Cloud Migration Strategy: A Comprehensive Overview

Unlock the potential of AWS cloud migration with the 7 Rs strategy. Explore benefits, challenges, and expert guidance for seamless digital transformation.

February 28, 2024

by Chiragsinh Vaghela

· 4,421 Views · 1 Like

Controlling Cloud Infrastructure Costs: Tips and Tricks

Track resources, set policies, explore serverless options, use discounts and set up alerts for cost monitoring. Expert advice helps.

February 28, 2024

by Den Smyrnov

· 3,707 Views · 1 Like

Logging and Monitoring in AWS

Explore the importance of logging and monitoring in AWS, various options and best practices, and popular AWS services and tools that can help you achieve these goals.

February 28, 2024

by Aditya Bhuyan

· 4,457 Views · 1 Like

Architectural Insights: Designing Efficient Multi-Layered Caching With Instagram Example

Explore the concept of multi-layered caching from both architectural and development perspectives, focusing on real-world applications like Instagram.

February 27, 2024

by Arun Pandey

CORE

· 17,483 Views · 5 Likes

The Latest Data Engineering Topics