Data Engineering Resources

High Fidelity Data: Balancing Privacy and Usage

Explore High Fidelity (HiFi) Data features such as visual, population, statistical, and ownership integrity, and its use to balance privacy and usability.

August 29, 2024

by Zilong Tang

· 6,130 Views · 4 Likes

When (Not) to Write an Apache APISIX Plugin

This post aims to provide practical alternatives to writing a custom plugin, offering solutions you can quickly implement in your projects.

August 29, 2024

by Nicolas Fränkel

· 3,612 Views · 1 Like

Team-as-Code: How to Apply Platform Engineering to DevOps’ Coding Stage

Learn how secure Cloud Development Environments (CDEs) automate environment setup, personalize developer experiences, enforce security policies, and more.

August 29, 2024

by Laurent Balmelli, PhD

· 6,269 Views · 2 Likes

Utilizing Multiple Vectors and Advanced Search Data Model Design for City Data

Learn how to build, architect, and design complex unstructured data applications for vector databases utilizing Milvus, GenAI, LangChain, YoLo, and more.

August 29, 2024

by Tim Spann

CORE

· 6,613 Views · 4 Likes

The Ultimate Guide to Evaluate RAG System Components: What You Need to Know

The RAG system contains two core components: the retriever and the generator. This article introduces how to evaluate these two components.

August 29, 2024

by Usama Jamil

· 3,302 Views · 4 Likes

Java Concurrency: Visibility and Synchronized

In this post, gain insight into variable visibility between two threads and what happens when we change a variable that is shared.

August 28, 2024

by Emmanouil Gkatziouras

CORE

· 9,813 Views · 6 Likes

Advanced Techniques in Automated Threat Detection

Explore advanced automated threat detection using AI and machine learning to combat evolving cyber threats, streamline security, and enhance protection.

August 28, 2024

by Anastasios Arampatzis

· 4,423 Views · 1 Like

The Evolution of Conversational AI: Blending Determinism With Dynamism

Learn how the future of conversational AI lies in the seamless integration of deterministic workflows and dynamic, creative responses.

August 28, 2024

by Xavier Portilla Edo

CORE

· 4,979 Views · 2 Likes

Why Replace External Database Caches?

Although external caches are a great companion for reducing latencies, they often introduce more problems than benefits. Learn more in this post.

August 28, 2024

by Felipe Mendes

· 5,784 Views · 3 Likes

Documenting a Java WebSocket API Using Smart-Doc

This article will detail how to use Smart-Doc to generate Java WebSocket interface documentation and provide a complete example of a WebSocket server.

August 28, 2024

by sun yu

· 8,217 Views · 3 Likes

Effortless Concurrency: Leveraging the Actor Model in Financial Transaction Systems

Learn challenges of financial transaction systems and implement robust mechanisms to handle concurrency effectively to ensure system integrity and reliability.

August 28, 2024

by Nikita Melnikov

· 32,274 Views · 6 Likes

Enhancing Accuracy in AI-Driven Mobile Applications: Tackling Hallucinations in Large Language Models

LLMs often hallucinate, leading to inaccurate responses. Contextualization and integrating knowledge bases can significantly reduce these errors.

August 27, 2024

by Nakul Pandey

· 3,383 Views · 1 Like

Are You Tired of Fragile Tests? Meet data-testid

Explore the data-testid attribute, which is specifically designed for testing purposes, and offers numerous advantages, particularly from a QA perspective.

August 27, 2024

by Shivam Bharadwaj

· 4,222 Views · 3 Likes

Building a Powerful AI and Machine Learning Pipeline: Best Practices and Tools

Explore best practices and steps for constructing a robust AI/ML pipeline, from data collection and processing to model deployment and monitoring.

August 27, 2024

by Kishorebabu Tenneti

· 4,193 Views · 2 Likes

LangChain Language Correctness Detector

Learn how to detect grammatical errors, sentiment, and aggressiveness in text using Langchain and OpenAI or Google Cloud language models.

August 27, 2024

by Xavier Portilla Edo

CORE

· 4,000 Views · 1 Like

Beyond the Obvious: Uncovering the Hidden Challenges in Cybersecurity

Explore often-overlooked cybersecurity challenges, from the human element to AI risks, as revealed by industry experts at Black Hat 2024.

August 27, 2024

by Tom Smith

CORE

· 5,382 Views · 1 Like

Telemetry Pipelines Workshop: Integrating Fluent Bit With OpenTelemetry, Part 1

Take a look at integrating Fluent Bit with OpenTelemetry for use cases where this connectivity in your telemetry data infrastructure is essential.

August 27, 2024

by Eric D. Schabell

CORE

· 5,886 Views · 4 Likes

Debugging Low Cache Hit Ratio

Explore debug scenarios where we can’t utilize cached data and need to read from the physical drive, why it's important, what to look for, and helpful tools.

August 27, 2024

by Adam Furmanek

CORE

· 2,747 Views · 3 Likes

Maximizing Enterprise Data: Unleashing the Productive Power of AI With the Right Approach

Learn how organizations can unlock the full potential of their AI initiatives by considering critical elements such as infrastructure, cost, and more.

August 26, 2024

by Pan Singh Dhoni

· 5,330 Views · 1 Like

Methodcentipede

See an example of an antipattern that can lead to difficulties in maintaining and testing code and approaches that allow you to structure your work in a preventative way.

August 26, 2024

by Anton Belyaev

· 9,256 Views · 3 Likes

The Latest Data Engineering Topics