Data Engineering Resources

Teradata Performance and Skew Prevention Tips

Use unique identifiers such as Teradata Primary Index to evenly distribute data across AMPs to improve database performance by 70%.

February 4, 2025

by Sudheer Kumar Lagisetty

· 4,360 Views · 10 Likes

Leveraging Neo4j for Effective Identity Access Management

Discussing Neo4j for IAM data modeling, the use cases, the guidelines and best practices, and how Neo4j can be integrated with existing IAM systems and tools.

February 4, 2025

by Ramesh Manickavel

CORE

· 4,467 Views · 2 Likes

A Guide to Automating AWS Infrastructure Deployment

AWS CloudFormation automates infrastructure with code, supporting Elastic Beanstalk, serverless, EC2, and more for consistent deployments.

February 4, 2025

by Praveen Kumar Thopalle

· 5,385 Views · 2 Likes

Data Governance Essentials: Policies and Procedures (Part 6)

Learn how data quality, policies, and procedures strengthen data governance by ensuring accuracy, compliance, and security for better decision-making.

February 4, 2025

by Sukanya Konatam

· 5,631 Views · 4 Likes

Processing Cloud Data With DuckDB And AWS S3

DuckDB's ability to read data directly from cloud storage, such as AWS S3, makes it particularly powerful for modern data architectures.

February 4, 2025

by Anil Kumar Moka

· 14,430 Views

SOC 2 Made Simple: Your Guide to Certification

This guide breaks down the SOC 2 Type 2 certification process into practical steps, from preparation to the audit, with some tips and tools to make the journey smoother.

February 4, 2025

by Roman Misyurin

· 4,079 Views · 3 Likes

4 Reasons Agentic AI Is Reshaping Enterprise Search

Discover how Agentic AI is transforming enterprise search, empowering smarter decisions, and enhancing self-service success.

February 4, 2025

by Taranjeet Singh

· 3,632 Views · 2 Likes

Build Modern Data Architectures With Azure Data Services

Modern data architecture with Azure enables scalable, secure, and efficient data management, real-time analytics, and business innovation.

February 4, 2025

by Aravind Nuthalapati

CORE

· 35,552 Views · 1 Like

Limitations of LLM Reasoning

This article provides an analysis of the reasoning limitations that exist in today's LLMs and the approaches to overcome them.

February 4, 2025

by Bhala Ranganathan

CORE

· 6,670 Views · 3 Likes

Building a Full-Stack Resume Screening Application With AI

In this article, you'll learn how to build a smart resume analyzing application using the DeepSeek R1 open-source GenAI LLM model and save operation costs.

February 3, 2025

by Anujkumarsinh Donvir

CORE

· 9,968 Views · 9 Likes

Java Stream API: 3 Things Every Developer Should Know About

Java Stream API simplifies collection processing with lazy evaluation, parallelism, and functional programming. Use it to write cleaner, efficient, and scalable code.

February 3, 2025

by Danil Temnikov

CORE

· 15,625 Views · 16 Likes

Best Practices for API Rate Limits and Quotas

Rate limits protect your infrastructure, and quotas help you monetize your APIs. Both are key parts of a healthy API strategy.

February 3, 2025

by Derric Gilling

CORE

· 6,587 Views · 1 Like

Managing Distributed System Locks With Azure Storage

Locks in distributed systems give processes exclusive access to resources. Timeout and lease-based locks help prevent deadlocks and resource contention.

February 3, 2025

by Siri Varma Vegiraju

CORE

· 11,281 Views · 1 Like

All You Need to Know About Apache Spark

Apache Spark is a fast, open-source cluster computing framework for big data, supporting ML, SQL, and streaming. It’s scalable, efficient, and widely used.

February 3, 2025

by Abhishek Trehan

· 4,259 Views · 2 Likes

90% Cost Reduction With Prefix Caching for LLMs

Up to 70% of prompts in LLM applications are repetitive. Prefix caching can reduce inference costs by up to 90%, thus optimizing performance and saving money.

February 3, 2025

by Mahak Shah

· 4,702 Views

Building Neural Networks With Automatic Differentiation

In this post, we will write a basic DNN using simple Python. To do that, we need to understand automatic differentiation and then implement it in code.

February 3, 2025

by Mayank Gupta

· 2,887 Views · 2 Likes

Building RAG Apps With Apache Cassandra, Python, and Ollama

A brief introduction to Apache Cassandra for retrieval-augmented generation using Python and Ollama for developing applications free of cost locally or on a server.

February 3, 2025

by Varun Setia

· 4,456 Views

Pydantic: Simplifying Data Validation in Python

Pydantic is a powerful Python library that uses type annotations to validate data structures. Learn about the powerful features of Pydantic with code examples.

February 3, 2025

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 3,394 Views · 6 Likes

The Quest for HA and DR in Loki

Minimize data loss and business disruption by implementing high availability and configuring disaster recovery for Loki with AWS S3 as the object store.

January 31, 2025

by Pavan N G

· 4,124 Views · 1 Like

Building Custom Tools With Model Context Protocol

Learn how to build MCP servers to extend AI capabilities. Create tools that AI models can seamlessly integrate, demonstrated through an arXiv paper search implementation.

January 31, 2025

by Aditya Karnam Gururaj Rao

· 10,439 Views · 4 Likes

The Latest Data Engineering Topics