Big Data Resources

Another Lifesaver for Data Engineers: Apache Doris Job Scheduler for Task Automation

The built-in Doris Job Scheduler triggers pre-defined operations efficiently and reliably. It is useful in many cases including ETL and data lake analytics.

June 17, 2024

by Shirley H.

· 2,028 Views · 1 Like

Build an ELT Pipeline From MySQL Database

Easily set up MySQL Change Data Capture using open-source Airbyte and Debezium to construct a near real-time ELT pipeline.

June 17, 2024

by Thalia Barrera

· 3,415 Views · 2 Likes

The Past, Present, and Future of Stream Processing

Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI, and Apache Iceberg.

June 17, 2024

by Kai Wähner

CORE

· 9,342 Views · 7 Likes

Explore the Complete Guide to Various Internet of Things (IoT) Protocols

As the IoT industry continues to evolve, it brings about revolutionary changes in device communication, further underscoring the importance of IoT protocols.

June 17, 2024

by BINU SUDHAKARAN PILLAI

· 12,333 Views · 5 Likes

Custom Health Checks in Spring Boot

Spring Boot provides health indicators to monitor application health and database health with the help of Spring Boot Actuator.

June 16, 2024

by Sai Krishna Reddy Chityala

· 14,989 Views · 5 Likes

How Sigma Is Empowering Devs, Engineers, and Architects With Cloud-Native Analytics

Sigma CEO Mike Palmer reveals how the company's cloud-native platform is transforming data analytics for developers, engineers, and architects.

June 13, 2024

by Tom Smith

CORE

· 4,971 Views · 1 Like

Why Use Rust Over C++ for IoT Solution Development

Rust is becoming popular for IoT development due to its superior security, efficient memory management, and modern syntax, overtaking C++.

June 12, 2024

by Den Smyrnov

· 5,085 Views · 1 Like

How Snowflake Is Powering the Future of Big Data With Apache Iceberg and Polaris

Snowflake's Ron Ortloff reveals how the data cloud company is accelerating big data analytics with Apache Iceberg and the new Polaris cloud service.

June 12, 2024

by Tom Smith

CORE

· 3,374 Views · 1 Like

KNIME’s Path To Empowering Developers in the Evolving Data Science Landscape

Michael Berthold, Founder and CEO of KNIME, shares insights on the company's evolution, addressing current data science challenges, and empowering developers.

June 11, 2024

by Tom Smith

CORE

· 1,668 Views · 1 Like

Exploring the Dynamics of Streaming Databases

This article explores what a streaming database is, as it is the core component of a stream processing system, and describes some available solutions.

June 11, 2024

by Federico Trotta

· 5,857 Views · 3 Likes

Choosing the Optimal Data Storage Architecture

Explore the differences between a Data Warehouse, Data Lake, and Data Lakehouse architectures to choose the best fit for your business needs.

June 10, 2024

by Den Smyrnov

· 2,594 Views · 1 Like

JavaScript, Node.js, and Apache Kafka for Full-Stack Data Streaming

Explore JavaScript, Node.js, and Apache Kafka for full-stack development for data streaming with open-source clients.

June 10, 2024

by Kai Wähner

CORE

· 7,421 Views · 2 Likes

Twilio Empowers Developers With Enhanced Data Warehouse Interoperability

Linked Audiences and Segment Data Graph revolutionize customer engagement by unlocking the full potential of data warehouses for developers and marketers.

June 10, 2024

by Tom Smith

CORE

· 2,496 Views · 1 Like

Using AWS Data Lake and S3 With SQL Server: A Detailed Guide With Research Paper Dataset Example

The integration of AWS Data Lake and Amazon S3 with SQL Server provides the ability to store data at any scale and leverage advanced analytics capabilities.

June 7, 2024

by Vijay Panwar

CORE

· 7,187 Views · 1 Like

Comprehensive Guide to Data Analysis and Visualization With Pandas and Matplotlib

Covering data preparation, descriptive statistics, profit margin calculations, and various types of plots such as bar charts, pie charts, and scatter plots.

June 7, 2024

by Krishnamurty Raju Mudunuru

· 7,647 Views · 1 Like

The Role of Data Engineering in Building Scalable AI Systems

Explore the role of data engineering in the field of artificial intelligence, specifically how data engineering influences the success and scalability of AI projects.

June 7, 2024

by Karthik Rajashekaran

CORE

· 3,622 Views · 2 Likes

ChatGPT: Your Guide to SQL Query Translation Between Databases

ChatGPT offers a solution for SQL translation by accurately translating queries, handling data types, and suggesting alternatives for proprietary features.

June 7, 2024

by Dmitry Narizhnykh

CORE

· 3,101 Views · 2 Likes

Building a Tool To Generate Text With OpenAI’s GPT-4 Model

Build a text generation tool using OpenAI's GPT-4. Set up your environment, authenticate the API, make API calls, handle errors, and integrate with Flask for a web app.

June 6, 2024

by Neha Dhaliwal

· 4,470 Views · 4 Likes

Streamlining Data Integration

This article demonstrates integrating Salesforce and Oracle into Amazon Redshift using Python and stored procedures, leveraging the ELT approach.

June 6, 2024

by Rajesh Remala

· 5,021 Views · 2 Likes

Take Your First Steps for Building on LLMs With Google Gemini

Learn to build an LLM application using the Google Gemini API and deploy it to Heroku. This guide walks you through setup, code, and deployment step-by-step.

June 5, 2024

by Alvin Lee

CORE

· 6,338 Views · 2 Likes

The Latest Big Data Topics