Data Engineering Resources

Build a Stateless Microservice With GitHub Copilot in VSCode

In this guide, we will build a stateless Flask Microservice using GitHub Copilot, with its benefits, speed, and where to be cautious when using it.

February 5, 2025

by Sibasis Padhi

· 8,943 Views · 7 Likes

Teradata Performance and Skew Prevention Tips

Use unique identifiers such as Teradata Primary Index to evenly distribute data across AMPs to improve database performance by 70%.

February 4, 2025

by Sudheer Kumar Lagisetty

· 4,486 Views · 10 Likes

Leveraging Neo4j for Effective Identity Access Management

Discussing Neo4j for IAM data modeling, the use cases, the guidelines and best practices, and how Neo4j can be integrated with existing IAM systems and tools.

February 4, 2025

by Ramesh Manickavel

CORE

· 4,675 Views · 2 Likes

A Guide to Automating AWS Infrastructure Deployment

AWS CloudFormation automates infrastructure with code, supporting Elastic Beanstalk, serverless, EC2, and more for consistent deployments.

February 4, 2025

by Praveen Kumar Thopalle

· 5,538 Views · 2 Likes

Data Governance Essentials: Policies and Procedures (Part 6)

Learn how data quality, policies, and procedures strengthen data governance by ensuring accuracy, compliance, and security for better decision-making.

February 4, 2025

by Sukanya Konatam

· 5,919 Views · 4 Likes

Processing Cloud Data With DuckDB And AWS S3

DuckDB's ability to read data directly from cloud storage, such as AWS S3, makes it particularly powerful for modern data architectures.

February 4, 2025

by Anil Kumar Moka

· 14,691 Views

SOC 2 Made Simple: Your Guide to Certification

This guide breaks down the SOC 2 Type 2 certification process into practical steps, from preparation to the audit, with some tips and tools to make the journey smoother.

February 4, 2025

by Roman Misyurin

· 4,207 Views · 3 Likes

4 Reasons Agentic AI Is Reshaping Enterprise Search

Discover how Agentic AI is transforming enterprise search, empowering smarter decisions, and enhancing self-service success.

February 4, 2025

by Taranjeet Singh

· 3,710 Views · 2 Likes

Build Modern Data Architectures With Azure Data Services

Modern data architecture with Azure enables scalable, secure, and efficient data management, real-time analytics, and business innovation.

February 4, 2025

by Aravind Nuthalapati

CORE

· 35,638 Views · 1 Like

Limitations of LLM Reasoning

This article provides an analysis of the reasoning limitations that exist in today's LLMs and the approaches to overcome them.

February 4, 2025

by Bhala Ranganathan

CORE

· 6,861 Views · 3 Likes

Building a Full-Stack Resume Screening Application With AI

In this article, you'll learn how to build a smart resume analyzing application using the DeepSeek R1 open-source GenAI LLM model and save operation costs.

February 3, 2025

by Anujkumarsinh Donvir

CORE

· 10,144 Views · 9 Likes

Java Stream API: 3 Things Every Developer Should Know About

Java Stream API simplifies collection processing with lazy evaluation, parallelism, and functional programming. Use it to write cleaner, efficient, and scalable code.

February 3, 2025

by Danil Temnikov

CORE

· 15,807 Views · 16 Likes

Best Practices for API Rate Limits and Quotas

Rate limits protect your infrastructure, and quotas help you monetize your APIs. Both are key parts of a healthy API strategy.

February 3, 2025

by Derric Gilling

CORE

· 6,912 Views · 1 Like

Managing Distributed System Locks With Azure Storage

Locks in distributed systems give processes exclusive access to resources. Timeout and lease-based locks help prevent deadlocks and resource contention.

February 3, 2025

by Siri Varma Vegiraju

CORE

· 11,471 Views · 1 Like

All You Need to Know About Apache Spark

Apache Spark is a fast, open-source cluster computing framework for big data, supporting ML, SQL, and streaming. It’s scalable, efficient, and widely used.

February 3, 2025

by Abhishek Trehan

· 4,401 Views · 2 Likes

90% Cost Reduction With Prefix Caching for LLMs

Up to 70% of prompts in LLM applications are repetitive. Prefix caching can reduce inference costs by up to 90%, thus optimizing performance and saving money.

February 3, 2025

by Mahak Shah

· 4,923 Views

Building Neural Networks With Automatic Differentiation

In this post, we will write a basic DNN using simple Python. To do that, we need to understand automatic differentiation and then implement it in code.

February 3, 2025

by Mayank Gupta

· 3,090 Views · 2 Likes

Building RAG Apps With Apache Cassandra, Python, and Ollama

A brief introduction to Apache Cassandra for retrieval-augmented generation using Python and Ollama for developing applications free of cost locally or on a server.

February 3, 2025

by Varun Setia

· 4,576 Views

Pydantic: Simplifying Data Validation in Python

Pydantic is a powerful Python library that uses type annotations to validate data structures. Learn about the powerful features of Pydantic with code examples.

February 3, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 3,555 Views · 6 Likes

The Quest for HA and DR in Loki

Minimize data loss and business disruption by implementing high availability and configuring disaster recovery for Loki with AWS S3 as the object store.

January 31, 2025

by Pavan N G

· 4,197 Views · 1 Like

The Latest Data Engineering Topics