Data Engineering Resources

Vibe Coding: Conversational Software Development — Part 4: Guiding AI Through Iteration

In part 4 of my 'Vibe Coding' series, learn more about conversational coding, code generation from prompts, and the future of developer productivity.

September 10, 2025

by Gaurav Gaur

CORE

· 3,457 Views · 2 Likes

Using Arrow Flight SQL to Improve Data Transfer Performance in Apache Doris

Learn how Arrow Flight SQL boosts Apache Doris data transfer up to 100x, enabling faster, efficient analysis and improved workflow in Python and Java.

September 10, 2025

by Zen Hua

· 2,060 Views · 1 Like

An Introduction to Artificial Intelligence: Neural Networks, NLP, and Word Embeddings

From a single neuron to understanding language, this article provides a short introduction to artificial intelligence for software engineers.

September 10, 2025

by Ioan Tinca

· 1,897 Views · 1 Like

Using ChartMuseum as a Helm Repository

ChartMuseum is a lightweight, open-source Helm Chart repository for Kubernetes. Learn how to deploy it, enable authentication, and integrate with CI/CD.

September 9, 2025

by Tanmay Batham

· 3,303 Views

Tutorial: RAG at Scale With Vector Databases vs Lakehouse Architectures

Learn how to scale RAG pipelines by storing embeddings in vector databases vs. lakehouses, with hands-on examples and key trade-offs.

September 9, 2025

by Vivek Venkatesan

· 3,309 Views

Understanding Table Statistics in SQL Server: Importance, Performance Impact, and Practical Examples

SQL Server table statistics guide the optimizer in building efficient query plans. DBAs must keep them updated to avoid poor performance from stale data.

September 9, 2025

by arvind toorpu

CORE

· 3,646 Views · 2 Likes

How to Use Jenkins Effectively With ECS/EKS Cluster

Jenkins on AWS ECS/EKS boosts CI/CD with scalability, cost-efficiency, and automation, using AI-driven dynamic scaling, security, and monitoring

September 9, 2025

by Bibin Skaria

· 3,015 Views · 2 Likes

Probably Secure: A Look at the Security Concerns of Deterministic vs Probabilistic Systems

Learn why deterministic security remains essential in an AI-driven world and how to combine probability and proof for safe, auditable development with LLMs.

September 9, 2025

by Dwayne McDaniel

· 1,666 Views · 1 Like

Is Your AI a Psychopath?

AI’s bugs aren’t code errors but symptoms of a broken 'mind.' LLMs show psychopathic traits: fragmented selves, weak brakes, and reward obsession.

September 9, 2025

by Taras Baranyuk

CORE

· 2,793 Views

Cloud Automation Excellence: Terraform, Ansible, and Nomad for Enterprise Architecture

Enterprise cloud architecture demands sophisticated orchestration of infrastructure, configuration, and workload management across diverse computing platforms.

September 9, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 5,474 Views · 3 Likes

Toward Explainable AI (Part 8): Bridging Theory and Practice—SHAP: Powerful, But Can We Trust It?

Explainable AI bridges the gap between complex models and real-world accountability, helping teams build trust, ensure compliance, and make smarter decisions.

September 9, 2025

by Frederic Jacquet

CORE

· 1,469 Views · 2 Likes

Replacing LEADTOOLS Scanner With AWS Textract (Step-by-Step Migration)

LEADTOOLS is a scanning tool that needs to be migrated to the AWS Cloud using an equivalent service, such as AWS Textract, for real-time document scanning.

September 8, 2025

by Prabhakar Mishra

· 2,509 Views · 1 Like

Stop Your GenAI From Burning Cash in Production

GenAI in production is expensive, but most teams waste 60-80% of their budget on preventable mistakes. Five proven optimizations that cut costs by 40-75%

September 8, 2025

by Praveen Chinnusamy

· 24,057 Views · 4 Likes

Building an AI-Powered Insurance Q and A Assistant With RAG and Snowflake Cortex

This project shows how to create an AI-powered insurance Q&A assistant using Retrieval-Augmented Generation and Snowflake Cortex Search for accurate answers.

September 8, 2025

by munikrishnaiah sundararamaiah

· 1,388 Views · 2 Likes

API Design First: AsyncAPI in .Net

AsyncAPI isn't as widely adopted as OpenAPI Spec, however, it's getting significant attention in the world of everything async and distributed.

September 8, 2025

by Shashi Kumar

· 2,343 Views · 2 Likes

Getting Started With ClickHouse for AI/ML in Python

ClickHouse is a high-performance, column-oriented OLAP database designed to handle petabyte-scale data with lightning-fast query execution.

September 8, 2025

by Devdas Gupta

· 2,289 Views · 3 Likes

Building a Platform Abstraction for AWS Networks Using Crossplane

Build Crossplane abstractions that let your devs request AWS networks without writing infrastructure code. Create XRD and composition to hide details.

September 5, 2025

by Ramesh Sinha

· 3,048 Views · 1 Like

Meta Prompting for Agile Practitioners

Learn how meta prompting turns AI into a partner for Retrospective prep—shaping context-rich dialogue while keeping facilitation a human skill.

September 5, 2025

by Stefan Wolpers

CORE

· 2,271 Views · 3 Likes

Making String Search Easier Across Databases

Users rarely remember exact strings. Partial search with fragments—beginnings, endings, or keywords—has become crucial for systems like e-commerce and finance.

September 5, 2025

by Otavio Santana

CORE

· 3,281 Views · 2 Likes

Measuring What Matters: A Strategic Lens on Transformation Metrics

Track Agile-DevOps and AI-first transformations effectively by selecting the right metrics—balancing output/outcome, leading/lagging, and subjective/objective measures.

September 5, 2025

by Suman Das

· 2,693 Views · 2 Likes

The Latest Data Engineering Topics