Data Engineering Resources

Discover the benefits of GitOps, explore a detailed comparison between ArgoCD and FluxCD, and learn how these tools streamline deployments and DevOps workflows.

July 30, 2024

by Unnati Mishra

· 7,803 Views · 4 Likes

How to Run Apache Spark on Kubernetes in Less Than 5 Minutes

Learn how to quickly set up and run Apache Spark on Kubernetes using Ilum for easier deployment, management, and scaling.

July 30, 2024

by Adam Lichtenstein

· 10,140 Views · 10 Likes

Implementing Multi-Level Caching in Java With NCache

Understanding the benefits of multi-level caching and setting up multi-level caching in Java using NCache — using it both as a distributed cache and a local cache.

July 30, 2024

by Abhinav Pandey

· 6,372 Views · 1 Like

Real-Time Anomaly Detection Using Large Language Models

Real-time anomaly detection using LLMs enhances accuracy for finance, healthcare, and cybersecurity through contextual analysis and pattern recognition.

July 30, 2024

by Harsh Daiya

CORE

· 12,393 Views · 3 Likes

Apache Iceberg Table Management

Apache Iceberg Table Management is essential to ensure smooth table operations by expiring/deleting unused data, metadata files.

July 30, 2024

by Rohit Kumar

· 3,150 Views · 1 Like

Enhancing Stream Data Processing With Snow Pipe, Cortex AI, and Snow Park

Customer sentiment analysis from reviews gathered from external source streams is ingested to Snowflake using Snow Pipe for further analysis using cortex AI API’s.

July 30, 2024

by Harshavardhan Yedla

· 3,608 Views · 3 Likes

Pivoting Database Systems Practices to AI: Create Efficient Development and Maintenance Practices With Generative AI

Explore how AI can revolutionize modern database development and maintenance through automation, real-time AI applications, and more.

July 30, 2024

by Anandaganesh Balakrishnan

· 7,280 Views · 4 Likes

How Kubernetes Changed the Networking Model and What Developers Should Know about eBPF and Cilium

Explore how Kubernetes transformed networking, the impact of eBPF and Cilium, and the future of network administration in this interview with Nicolas Vibert.

July 30, 2024

by Tom Smith

CORE

· 16,683 Views · 5 Likes

The Future of AI: Exploring Generative Systems and Large Language Models

Unleashing creativity and solving complex problems: the promise and challenges of generative AI and large language models.

July 30, 2024

by Saigurudatta Pamulaparthyvenkata

· 2,064 Views · 2 Likes

Using SingleStore for Iceberg Catalog Storage

SingleStore recently announced bi-directional support for Apache Iceberg. In this short article, we'll implement an Iceberg catalog using SingleStore and JDBC.

July 29, 2024

by Akmal Chaudhri

CORE

· 4,665 Views · 1 Like

Calm Down, Cloud's Not THAT Different!

Many network folks find the idea of learning cloud architecture daunting. The truth is that the two are more similar than they seem.

July 29, 2024

by Leon Adato

· 2,692 Views · 2 Likes

AIGenOps: Generative AI and Platform Engineering

In the domain of regulated software, especially in the banking environment where we work, there are constraints of security, quality, and network limitations.

July 29, 2024

by Nicolas Fantoni

· 3,327 Views · 6 Likes

Guarding Privacy: Cutting-Edge Technologies for Data Protection

Discover how advanced technologies are revolutionizing data security. Learn about encryption, blockchain, and machine learning in the fight against cyber threats.

July 29, 2024

by Saigurudatta Pamulaparthyvenkata

· 3,558 Views · 2 Likes

Automating Databases for Modern DevOps Practices: A Guide to Common Patterns and Anti-Patterns for Database Automation Techniques

By embracing AI, automation, database DevOps, and other emerging trends, organizations can ensure robust, efficient, and future-proofed database management practices.

July 29, 2024

by Naga Santhosh Reddy Vootukuri

CORE

· 7,693 Views · 9 Likes

Application-Level Tracing: The Good, the Bad, and the Alternative

Traditional app-level tracing was good up until a certain level. For complex troubleshooting, live debugging is taking the first seat in modern observability.

July 29, 2024

by Eran Kinsbruner

· 2,814 Views · 4 Likes

Implementing and Deploying a Real-Time AI-Powered Chatbot With Serverless Architecture

Build a real-time AI chatbot using AWS Lambda for the backend and a simple HTML/JavaScript frontend.

July 29, 2024

by Neha Dhaliwal

· 4,880 Views · 3 Likes

Enhancing Query Performance With AI and Vector Search in Azure Cosmos DB for PostgreSQL

Learn to enhance query performance using AI and vector search in Azure Cosmos DB for PostgreSQL, boosting data retrieval efficiency and accuracy.

July 29, 2024

by Sameer Danave

· 2,238 Views · 2 Likes

Why OOP Is a Bad Fit for Custom Software

Everything in programming is a trade-off. The combination of imperative and (DO)RY is the best one that results in more total code, but more maintainable code.

July 29, 2024

by Greg Hall

· 7,802 Views · 2 Likes

DB2 vs. Oracle for IBM Maximo: A Comparative Analysis

This essay explores the differences between DB2 and Oracle in the context of their integration with IBM Maximo and specific features that benefit Maximo environments.

July 28, 2024

by Lakshmana Rao Koppada

· 3,503 Views · 1 Like

Real-Time Streaming Architectures: A Technical Deep Dive Into Kafka, Flink, and Pinot

While individual components like Kafka, Flink, and Pinot are very powerful, managing them at scale across cloud and on-premises deployments can be operationally complex.

July 28, 2024

by Abhishek Gupta

CORE

· 10,843 Views · 4 Likes

The Latest Data Engineering Topics