Data Engineering Resources

Unmasking the Danger: 10 Ways AI Can Go Rogue (And How to Spot Them)

Large language models pose risks like manipulation, cyberattacks, and unintended self-improvement, but safeguards are being developed.

June 18, 2024

by Indrajit Bhattacharya

· 2,829 Views · 1 Like

Harnessing Kafka Streams for Seamless Data Tasks

Learn about the use of Kafka for convenient data exploration and dynamic data integration with its "publish-subscribe" model, organizing data into topics.

June 18, 2024

by Ilia Ivankin

· 5,027 Views · 2 Likes

Random and Fixed Routes With Apache APISIX

Some triage on the APISIX repo led to the discovery of an interesting use case involving random and fixed routes. Explore the solution in this post.

June 18, 2024

by Nicolas Fränkel

· 2,572 Views · 1 Like

Another Lifesaver for Data Engineers: Apache Doris Job Scheduler for Task Automation

The built-in Doris Job Scheduler triggers pre-defined operations efficiently and reliably. It is useful in many cases including ETL and data lake analytics.

June 17, 2024

by Shirley H.

· 2,092 Views · 1 Like

Using a Body With an HTTP Get Method Is Still a Bad Idea

Learn why it's technically possible but generally not recommended to send a body with an HTTP GET request.

June 17, 2024

by Danny Logsdon

· 13,240 Views · 10 Likes

Build an ELT Pipeline From MySQL Database

Easily set up MySQL Change Data Capture using open-source Airbyte and Debezium to construct a near real-time ELT pipeline.

June 17, 2024

by Thalia Barrera

· 3,493 Views · 2 Likes

Sales Forecasting With Snowflake Cortex ML Functions

Snowflake Cortex is a collection of integrated machine learning models and AI features designed to implement AI-driven solutions within the Snowflake environment.

June 17, 2024

by Kapil Kumar Sharma

· 4,197 Views · 2 Likes

Advancements in Machine Learning Classification Techniques for Data Quality Improvement

This article is an analysis of how ML classification techniques help improve data quality and lead to better customer data insights.

June 17, 2024

by Akshay Agarwal

· 5,339 Views · 4 Likes

The Past, Present, and Future of Stream Processing

Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI, and Apache Iceberg.

June 17, 2024

by Kai Wähner

CORE

· 9,449 Views · 7 Likes

How AI Agentic Workflows Could Drive More AI Progress Than Even the Next Generation of Foundation Models

Dr. Andrew Ng discusses how AI agentic workflows could revolutionize AI development, surpassing the impact of next-generation foundation models.

June 17, 2024

by Tom Smith

CORE

· 3,067 Views · 1 Like

How Retrieval Augment Generation Makes LLMs Smarter Than Before

By feeding LLMs the necessary domain knowledge, prompts can be given context and yield better results. RAG can decrease hallucination along with several other advantages.

June 17, 2024

by Kevin Vu

· 2,038 Views · 1 Like

Explore the Complete Guide to Various Internet of Things (IoT) Protocols

As the IoT industry continues to evolve, it brings about revolutionary changes in device communication, further underscoring the importance of IoT protocols.

June 17, 2024

by BINU SUDHAKARAN PILLAI

· 12,450 Views · 5 Likes

Custom Health Checks in Spring Boot

Spring Boot provides health indicators to monitor application health and database health with the help of Spring Boot Actuator.

June 16, 2024

by Sai Krishna Reddy Chityala

· 15,359 Views · 5 Likes

The Builder Pattern in Java: Guide to Test Data Generation

In this tutorial, learn how to use the Builder pattern in Java with a Datafaker library to generate test data for automated testing.

June 15, 2024

by Faisal Khatri

CORE

· 14,553 Views · 9 Likes

Building Better on AWS With the Enhanced AWS Well-Architected Framework

This blog dives into the recent enhancements to the AWS Well-Architected Framework, exploring how these updates can empower you to build even better on AWS.

June 14, 2024

by Kishore Babu Karumanchi

· 4,769 Views · 5 Likes

Oxide Computer: Revolutionizing Cloud Computing for the Modern Data Center

Discover how Oxide Computer transforms the data center landscape with their innovative cloud computing solutions designed for the modern enterprise.

June 14, 2024

by Tom Smith

CORE

· 4,357 Views · 1 Like

Building an Internal TLS and SSL Certificate Monitoring Agent: From Concept to Deployment

Learn how an internal SSL/TLS certificate monitoring agent was built, including requirements, architecture, scheduling, integrations, and UI.

June 14, 2024

by Max Shash

CORE

· 8,717 Views · 4 Likes

Docker + .NET APIs: Simplifying Deployment and Scaling

This article explores the benefits of using Docker containers with .NET applications and provides a step-by-step guide to getting started.

June 13, 2024

by Naga Santhosh Reddy Vootukuri

CORE

· 8,438 Views · 3 Likes

Fine-Tuning Performance, Resolving Common Issues in FinTech Application With MySQL

Optimize MySQL Performance: Avoid complex views, full table scans, wildcard searches, and batch inserts. Use connection labels, purge data, and leverage indexes.

June 13, 2024

by Amol Gote

CORE

· 5,917 Views · 4 Likes

How Sigma Is Empowering Devs, Engineers, and Architects With Cloud-Native Analytics

Sigma CEO Mike Palmer reveals how the company's cloud-native platform is transforming data analytics for developers, engineers, and architects.

June 13, 2024

by Tom Smith

CORE

· 5,129 Views · 1 Like

The Latest Data Engineering Topics