Data Engineering Resources

Designing a Java Connector for Software Integrations

A Java connector unites different systems by allowing them to send information effectively while making crucial data freely available through interoperable interfaces.

May 15, 2025

by Anant Wairagade

· 7,504 Views · 1 Like

Vibe Coding With GitHub Copilot: Optimizing API Performance in Fintech Microservices

Can GitHub Copilot optimize fintech APIs? We test its performance, help, vibe coding flow, real-world impact, and limits.

May 15, 2025

by Sibasis Padhi

· 8,305 Views · 8 Likes

Customer 360: Fraud Detection in Fintech With PySpark and ML

A big data ML solution using PySpark to link bank accounts and transactions for a unified customer view (Customer 360), while detecting fraud implicitly.

May 15, 2025

by Ram Ghadiyaram

CORE

· 4,858 Views · 7 Likes

Mastering Advanced Aggregations in Spark SQL

OLAP (Online Analytical Processing) aggregation techniques in Spark SQL are used for complex, multi-level data analysis over large datasets.

May 15, 2025

by Ram Ghadiyaram

CORE

· 5,456 Views · 8 Likes

Cosmos DB Disaster Recovery: Multi-Region Write Pitfalls and How to Evade Them

Learn how multi-region writes ad managed in Cosmos DB, key considerations to steer clear or, and best practices to architect a resilient system.

May 14, 2025

by Yash Gautam

· 5,113 Views · 3 Likes

Developers Beware: Slopsquatting and Vibe Coding Can Increase Risk of AI-Powered Attacks

Slopsquatting and vibe coding are fueling a new wave of AI-driven cyberattacks, exposing developers to hidden risks through fake, hallucinated packages.

May 14, 2025

by Aminu Abdullahi

· 4,747 Views · 2 Likes

FIPS 140-3: The Security Standard That Protects Our Federal Data

Learn how FIPS 140-3, NIST's latest cryptographic standard, enhances security with zero trust and encryption for protecting sensitive federal data.

May 14, 2025

by Gireesh Punathil

· 4,274 Views · 1 Like

Why Database Migrations Take Months and How to Speed Them Up

Database migrations are more than just moving data. With proper planning, preparation, and testing, you can reduce migration time to days or hours.

May 14, 2025

by Alexander Komyagin

· 4,437 Views · 1 Like

MySQL to PostgreSQL Database Migration: A Practical Case Study

In this case study, learn how migrating MySQL to PostgreSQL boosted query speed, optimized JSON and JOINs, and improved scalability for an e-commerce platform.

May 14, 2025

by Vlad Bilyak

· 4,879 Views · 2 Likes

Understanding IEEE 802.11(Wi-Fi) Encryption and Authentication: Write Your Own Custom Packet Sniffer

Learn about the most common encryption and authentication methods per the IEEE 802.11 standards, how to write your own packet sniffer and use Wireshark.

May 14, 2025

by Rahul Nagraj

· 6,127 Views · 2 Likes

Unlocking AI Coding Assistants: Generate Unit Tests

This blog post instructs on creating qualitative unit tests for a Spring Boot application using an AI coding assistant and its capabilities and limitations.

May 14, 2025

by Gunter Rotsaert

CORE

· 4,253 Views · 2 Likes

Beyond Code Coverage: A Risk-Driven Revolution in Software Testing With Machine Learning

Machine learning-driven, risk-based testing targets critical risks, automates test selection, and improves software reliability.

May 14, 2025

by Gopinath Kathiresan

· 4,143 Views · 2 Likes

A Simple, Convenience Package for the Azure Cosmos DB Go SDK

Learn about cosmosdb-go-sdk-helper: Simplify Azure Cosmos DB operations with Go. Features auth, queries, error handling, metrics, and Azure Functions support.

May 14, 2025

by Abhishek Gupta

CORE

· 3,822 Views · 1 Like

Intro to RAG: Foundations of Retrieval Augmented Generation, Part 1

Retrieval-augmented generation (RAG) retrieves relevant information from external sources to improve the accuracy and reliability of responses, making it a powerful tool.

May 14, 2025

by Jennifer Reif

CORE

· 3,539 Views · 3 Likes

Performing and Managing Incremental Backups Using pg_basebackup in PostgreSQL 17

PostgreSQL 17 adds incremental backups with pg_basebackup and pg_combinebackup, saving time and storage by capturing only changed data since the last backup.

May 14, 2025

by arvind toorpu

CORE

· 4,102 Views · 4 Likes

Ethical AI in Agile

Learn all about ethical AI integration for Agile teams with four key guardrails: Data Privacy, Human Value, Output Validation, and Transparent Attribution.

May 14, 2025

by Stefan Wolpers

CORE

· 2,876 Views · 2 Likes

A Deep Dive Into Firmware Over the Air for IoT Devices

Dive into FOTA, its importance, and best practices to design reliable and efficient FOTA services for IoT devices.

May 13, 2025

by Shweta Agarwal

· 3,820 Views · 1 Like

Agentic AI for Automated Application Security and Vulnerability Management

Explore how agentic AI enables several agents to jointly detect, fix, and verify security vulnerabilities in your codebase.

May 13, 2025

by Vamsi Kavuri

CORE

· 13,064 Views · 1 Like

Ensuring Configuration Consistency Across Global Data Centers

Global data centers need consistent configurations, but engineers misalign across regions. This paper explores the problem, existing solutions, and a proposed approach.

May 13, 2025

by Shubham Jindal

· 2,386 Views · 6 Likes

Fraud Detection Using Artificial Intelligence and Machine Learning

AI/ML is revolutionizing fraud detection with automated, adaptive, real-time anomaly detection, despite challenges like interpretability.

May 13, 2025

by Nikhil Kapoor

· 2,860 Views · 3 Likes

The Latest Data Engineering Topics