Data Engineering Resources

Efficient Data Management With Offset and Cursor-Based Pagination in Modern Applications

Explore offset and cursor-based pagination, integrated with Jakarta Data, Quarkus, and MongoDB, highlighting their benefits and practical use in REST APIs.

June 19, 2024

by Otavio Santana

CORE

· 7,129 Views · 3 Likes

Building Powerful AI Applications With Amazon Bedrock: Enhanced Chatbots and Image Generation Use Cases

This article delves into the challenges faced by Chatbots and Image Generation, showcasing how AWS Bedrock empowers businesses to overcome them.

June 19, 2024

by Kishore Babu Karumanchi

· 3,382 Views · 2 Likes

A Wake-up Call for Cloud Security: Insights From the Recent Snowflake Data Breach

A case study on the recent Snowflake data breach and deep dive into cloud security measures to fortify cloud environments.

June 19, 2024

by Venkata Nedunoori

· 3,053 Views · 1 Like

MLOps in Software-Defined Vehicles: A Centralized Platform Approach

MLOps platforms are essential for managing ML models in software-defined vehicles, ensuring their effectiveness and long-term success.

June 18, 2024

by Pavan Vemuri

· 2,122 Views · 4 Likes

Tobiko Data: Revolutionizing Data Transformation With SQLMesh

Discover how Tobiko Data's SQLMesh is transforming the data landscape, enabling data scientists and analysts to build efficient and correct pipelines.

June 18, 2024

by Tom Smith

CORE

· 2,905 Views · 1 Like

A Framework for Building Semantic Search Applications With Generative AI

This article explores a framework for building semantic search applications with generative AI, enabling developers with a unified scalable framework.

June 18, 2024

by Lakshmanan Sethu Sankaranarayanan

· 2,304 Views · 3 Likes

Unmasking the Danger: 10 Ways AI Can Go Rogue (And How to Spot Them)

Large language models pose risks like manipulation, cyberattacks, and unintended self-improvement, but safeguards are being developed.

June 18, 2024

by Indrajit Bhattacharya

· 2,179 Views · 1 Like

Harnessing Kafka Streams for Seamless Data Tasks

Learn about the use of Kafka for convenient data exploration and dynamic data integration with its "publish-subscribe" model, organizing data into topics.

June 18, 2024

by Ilia Ivankin

· 4,290 Views · 1 Like

Random and Fixed Routes With Apache APISIX

Some triage on the APISIX repo led to the discovery of an interesting use case involving random and fixed routes. Explore the solution in this post.

June 18, 2024

by Nicolas Fränkel

CORE

· 1,841 Views · 1 Like

Another Lifesaver for Data Engineers: Apache Doris Job Scheduler for Task Automation

The built-in Doris Job Scheduler triggers pre-defined operations efficiently and reliably. It is useful in many cases including ETL and data lake analytics.

June 17, 2024

by Shirley H.

· 1,551 Views · 1 Like

Using a Body With an HTTP Get Method Is Still a Bad Idea

Learn why it's technically possible but generally not recommended to send a body with an HTTP GET request.

June 17, 2024

by Danny Logsdon

· 7,419 Views · 9 Likes

Build an ELT Pipeline From MySQL Database

Easily set up MySQL Change Data Capture using open-source Airbyte and Debezium to construct a near real-time ELT pipeline.

June 17, 2024

by Thalia Barrera

· 2,767 Views · 2 Likes

Sales Forecasting With Snowflake Cortex ML Functions

Snowflake Cortex is a collection of integrated machine learning models and AI features designed to implement AI-driven solutions within the Snowflake environment.

June 17, 2024

by Kapil Kumar Sharma

· 3,162 Views · 2 Likes

Advancements in Machine Learning Classification Techniques for Data Quality Improvement

This article is an analysis of how ML classification techniques help improve data quality and lead to better customer data insights.

June 17, 2024

by Akshay Agarwal

· 3,941 Views · 4 Likes

The Past, Present, and Future of Stream Processing

Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI, and Apache Iceberg.

June 17, 2024

by Kai Wähner

CORE

· 8,246 Views · 7 Likes

How AI Agentic Workflows Could Drive More AI Progress Than Even the Next Generation of Foundation Models

Dr. Andrew Ng discusses how AI agentic workflows could revolutionize AI development, surpassing the impact of next-generation foundation models.

June 17, 2024

by Tom Smith

CORE

· 1,790 Views · 1 Like

How Retrieval Augment Generation Makes LLMs Smarter Than Before

By feeding LLMs the necessary domain knowledge, prompts can be given context and yield better results. RAG can decrease hallucination along with several other advantages.

June 17, 2024

by Kevin Vu

· 1,320 Views · 1 Like

Explore the Complete Guide to Various Internet of Things (IoT) Protocols

As the IoT industry continues to evolve, it brings about revolutionary changes in device communication, further underscoring the importance of IoT protocols.

June 17, 2024

by BINU SUDHAKARAN PILLAI

· 2,684 Views · 2 Likes