Data Engineering Resources

Bridging OT and IT: IIoT Middleware for Edge and Cloud With Kafka and Flink

Explore how IIoT middleware and data streaming tools like Kafka and Flink bridge OT and IT, enabling real-time integration and smarter industrial operations.

April 2, 2025

by Kai Wähner

CORE

· 4,690 Views · 4 Likes

How to Build a New API Quickly Using Spring Boot and Maven

Realizing that not all use Gradle, I thought I would demonstrate how quickly a RESTful API can be created by leveraging ChatGPT, Spring Boot, Maven, and Heroku.

April 1, 2025

by John Vester

CORE

· 51,285 Views · 15 Likes

Understanding SQL Server Query Store: A Comprehensive Guide

In this article, we will explore what the Query Store is, its benefits, scenarios for its use, and steps to implement it with examples.

April 1, 2025

by arvind toorpu

CORE

· 6,786 Views · 8 Likes

Building Scalable Data Lake Using AWS

Build a secure, scalable AWS data lake with S3, Glue, and Lake Formation, and learn key components, best practices, and analytics for optimized insights.

April 1, 2025

by Digvijay Waghela

· 4,813 Views · 3 Likes

Safeguarding Sensitive Data: Content Detection Technologies in DLP

Data breaches cost $4.88M on average — learn in this article how DLP content detection protects sensitive data with the help of AI, RegEx, OCR, and more.

April 1, 2025

by Praveen Kumar Myakala

· 7,224 Views · 44 Likes

Generative AI in Agile: A Strategic Career Decision

The cost of experimentation is low; the potential career advantage, on the other side, is substantial. A reason to embrace generative AI in Agile?

April 1, 2025

by Stefan Wolpers

CORE

· 5,156 Views · 6 Likes

Testing AI Is the First Step to Using AI

Sales and marketing professionals in high tech should use AI, but how do you start? There's a crucial first step that often gets skipped: testing.

April 1, 2025

by Jesse Casman

CORE

· 2,925 Views · 2 Likes

Predicting Ad Viewability With XGBoost Regressor Algorithm

This article explores how data-driven decision-making enhances digital advertising, focusing on predicting ad viewability rates using machine learning.

April 1, 2025

by MELTEM TOSUN

· 2,046 Views · 2 Likes

Testing DeepSeek-R1 Locally Using Ollama

Testing DeepSeek-R1 with Ollama for code generation, math solving, creative writing, and Q&A. See how it performs across tasks and where it falls short.

April 1, 2025

by Akmal Chaudhri

CORE

· 1,984 Views

Function Calling and Agents in Agentic AI

Function calling in agentic AI allows autonomous agents to dynamically execute tasks by invoking specific functions based on reasoning and goals.

April 1, 2025

by Bhala Ranganathan

CORE

· 3,219 Views · 1 Like

Understanding Database Consistency: A Key Concept in Distributed Systems

This article explores database consistency models in distributed systems and explains trade-offs between strong, eventual, causal, and other consistency types.

March 31, 2025

by Sandeep Kumar Gond

· 6,304 Views · 5 Likes

Doris vs Elasticsearch: A Comparison and Practical Cost Case Study

Apache Doris excels in analytics, SQL support, and cost efficiency, while Elasticsearch leads in text search but has higher storage costs and complexity.

March 31, 2025

by Darren Xu

· 3,828 Views · 2 Likes

Stop Exposing Secrets! Secure Your APIs in Postman Like a Pro

Don't expose your secrets! Learn how to secure your APIs in Postman with Vault, Secret Scanner, and best practices to keep your data safe.

March 31, 2025

by Gbadebo Bello

· 4,737 Views · 6 Likes

Supervised and Unsupervised Learning in Machine Learning

Discover the pros and cons, types, applications, and lots more about two common machine learning algorithms: supervised learning and unsupervised learning.

March 31, 2025

by Jennifer Wales

· 2,471 Views · 2 Likes

AWS WAF Classic vs WAFV2: Features and Migration Considerations

AWS WAFV2 is a major upgrade over WAF Classic, offering better scalability, flexibility, and automation. We'll also look at key migration considerations.

March 31, 2025

by Srinivas Chippagiri

CORE

· 20,651 Views · 2 Likes

How Does Video Annotation Augment Computer Vision?

Video annotation is the way in which machines are actually getting the ability to process visual data; this is what closes the gap between AI and real-world applications.

March 31, 2025

by Rayan Potter

· 1,980 Views · 1 Like

Product Design vs Platform Design for Software Development

Compare platform and product design approaches for business needs. Understand their differences and compare them to find the best fit for your use case.

March 31, 2025

by Ravi Kiran Mallidi

CORE

· 4,475 Views · 5 Likes

The Role of DevSecOps in Securing Multi-Cloud Architectures

Discover how DevSecOps enhances security in multi-cloud ecosystems, enabling safe implementation and addressing key challenges with effective strategies.

March 31, 2025

by Hrushikesh Deshmukh

· 4,553 Views · 2 Likes

Getting Started With LangChain for Beginners

This tutorial demonstrates how to use the LangChain framework to connect with OpenAI and other LLMs, work with various chains, and build a basic chatbot with history.

March 28, 2025

by Amit Chaudhary

· 5,302 Views · 5 Likes

Challenges of Using LLMs in Production: Constraints, Hallucinations, and Guardrails

By integrating post-processing validation, RAG, and customizable guardrails, developers can bridge the gap between prototyping and production in LLM applications.

March 28, 2025

by Shiv Kumar

· 3,290 Views · 1 Like

The Latest Data Engineering Topics