Data Engineering Resources

Enhanced API Security: Fine-Grained Access Control Using OPA and Kong Gateway

Kong secures APIs and OPA handles policy decisions. This custom plugin caches user data, minimizing database checks and ensuring fast, accurate access control.

October 4, 2024

by Amrutha Paladugu

· 7,192 Views · 5 Likes

How to Create a Search Engine and Algorithm With ClickHouse and Snowflake

Explore a step-by-step guide to developing a search engine and algorithm using Clickhouse, an open-source data warehousing solution.

October 3, 2024

by Akash Mukherjee

· 5,146 Views · 2 Likes

Implementing Polling With RxJS

Learn how to implement polling with RxJS for efficient, repetitive API calls, and optimize performance with exponential backoff and advanced RxJS operators.

October 3, 2024

by Aliaksandr Mirankou

· 8,137 Views · 1 Like

Why Doesn't Postgres Use My Index?

It may happen that we create an index, but PostgreSQL doesn’t use it. What could be the reason, and how can we fix it?

October 3, 2024

by Adam Furmanek

CORE

· 4,959 Views · 8 Likes

Supercharge Your Search With GenAI: From Simple Queries to Smarter Results

Boost your search with GenAI models for smarter, more relevant results using query expansion and reformulation using Chain of Thought prompting.

October 3, 2024

by Suruchi Shah

· 3,870 Views · 4 Likes

A Comprehensive Guide to Database Sharding: Building Scalable Systems

Explore an in-depth guide to database sharding: what it is, its types, how to select shard keys, and route queries for building scalable systems.

October 2, 2024

by Suleiman Dibirov

CORE

· 9,222 Views · 9 Likes

Hyperscale Data Centers Ignite Construction Surge

Demand is growing for robust computing. However, hyperscale data center construction faces challenges due to technology demands, shortages, and regulations.

October 2, 2024

by Emily Newton

· 5,246 Views · 1 Like

Key Considerations for Effective AI/ML Deployments in Kubernetes

Learn about managing AI/ML workloads in Kubernetes and follow a step-by-step tutorial for deploying a sample TensorFlow-based sentiment analysis model.

October 2, 2024

by Rajesh Gheware

CORE

· 9,831 Views · 2 Likes

Transforming Manufacturing: AI-Driven Quality Control and Predictive Maintenance Using Cloud and IoT

In modern manufacturing, the key to sustaining product reliability, compliance, and efficiency lies in robust quality control and advanced maintenance management.

October 2, 2024

by vuppulapati Chandra Sekhar Naidu

· 4,761 Views · 1 Like

Top 6 Cybersecurity Threat Detection Use Cases: How AI/ML Can Help Detect Advanced and Emerging Threats

Learn how to get started to design and build AI/ML solutions that are capable of revolutionizing the cybersecurity threat detection landscape.

October 2, 2024

by Mayank Singhi

· 4,151 Views · 2 Likes

AWS LetsEncrypt Lambda or Why I Wrote a Custom TLS Provider for AWS Using OpenTofu and Go

LetsEncrypt Lambda helps to manage TLS certificates. Compared to Certificate Manager it provides certs that can be used at non-only AWS services like EC2 Nginx.

October 2, 2024

by Alexander Sharov

· 7,685 Views · 3 Likes

Misconfiguration Madness: Thwarting Common Vulnerabilities in the Financial Sector

Financial services are among the most attacked sectors of any industry, making it critical that developers operate at the highest level to produce secure code.

October 2, 2024

by Matias Madou

· 4,404 Views · 2 Likes

How JavaScript DataTable Libraries Address Challenging Requirements in Web Data Management

This article highlights challenging project requirements on JS grid tables and how popular UI libraries help solve arising issues.

October 2, 2024

by Ivan Petrenko

· 4,874 Views · 1 Like

Unit Integration Testing With Testcontainers Docker Compose

Is your test dependent on multiple other applications? Do you want to create an integration test using Testcontainers? Learn about the Docker Compose Module.

October 2, 2024

by Gunter Rotsaert

CORE

· 6,011 Views · 1 Like

Leveraging Artificial Intelligence for Software Testing to Test Healthcare Applications

Learn how leveraging AI-powered software testing for healthcare domains can make a significant difference in the quality, efficiency, and reliability of care.

October 1, 2024

by Vikram Singh

· 4,647 Views · 5 Likes

How to Improve the Developer vs. AI Relationship

Less than half of software developers trust the accuracy of AI tools. Here are five ways that we can build a healthier relationship with AI.

October 1, 2024

by Knut Sveidqvist

· 3,232 Views · 2 Likes

Workarounds for Oracle Restrictions on the Size of Expression Lists

Learn five strategies to workaround ORA-01795 and ORA-00913 errors and explore a comparative analysis of their performance.

October 1, 2024

by Thiago Nascimento

CORE

· 8,384 Views · 4 Likes

Best Practices for Market Data Replay

Based on DolphinDB’s streaming replay, distributed database architecture, and APIs, you can create a tool for historical insight and model back-testing.

October 1, 2024

by Davis Zhou

· 3,369 Views · 1 Like