Big Data Resources

Choosing the Optimal Data Storage Architecture

Explore the differences between a Data Warehouse, Data Lake, and Data Lakehouse architectures to choose the best fit for your business needs.

June 10, 2024

by Den Smyrnov

· 2,561 Views · 1 Like

JavaScript, Node.js, and Apache Kafka for Full-Stack Data Streaming

Explore JavaScript, Node.js, and Apache Kafka for full-stack development for data streaming with open-source clients.

June 10, 2024

by Kai Wähner

CORE

· 7,371 Views · 2 Likes

Twilio Empowers Developers With Enhanced Data Warehouse Interoperability

Linked Audiences and Segment Data Graph revolutionize customer engagement by unlocking the full potential of data warehouses for developers and marketers.

June 10, 2024

by Tom Smith

CORE

· 2,459 Views · 1 Like

Using AWS Data Lake and S3 With SQL Server: A Detailed Guide With Research Paper Dataset Example

The integration of AWS Data Lake and Amazon S3 with SQL Server provides the ability to store data at any scale and leverage advanced analytics capabilities.

June 7, 2024

by Vijay Panwar

CORE

· 7,117 Views · 1 Like

Comprehensive Guide to Data Analysis and Visualization With Pandas and Matplotlib

Covering data preparation, descriptive statistics, profit margin calculations, and various types of plots such as bar charts, pie charts, and scatter plots.

June 7, 2024

by Krishnamurty Raju Mudunuru

· 7,606 Views · 1 Like

The Role of Data Engineering in Building Scalable AI Systems

Explore the role of data engineering in the field of artificial intelligence, specifically how data engineering influences the success and scalability of AI projects.

June 7, 2024

by Karthik Rajashekaran

CORE

· 3,582 Views · 2 Likes

ChatGPT: Your Guide to SQL Query Translation Between Databases

ChatGPT offers a solution for SQL translation by accurately translating queries, handling data types, and suggesting alternatives for proprietary features.

June 7, 2024

by Dmitry Narizhnykh

CORE

· 3,063 Views · 2 Likes

Building a Tool To Generate Text With OpenAI’s GPT-4 Model

Build a text generation tool using OpenAI's GPT-4. Set up your environment, authenticate the API, make API calls, handle errors, and integrate with Flask for a web app.

June 6, 2024

by Neha Dhaliwal

· 4,424 Views · 4 Likes

Streamlining Data Integration

This article demonstrates integrating Salesforce and Oracle into Amazon Redshift using Python and stored procedures, leveraging the ELT approach.

June 6, 2024

by Rajesh Remala

· 4,970 Views · 2 Likes

Take Your First Steps for Building on LLMs With Google Gemini

Learn to build an LLM application using the Google Gemini API and deploy it to Heroku. This guide walks you through setup, code, and deployment step-by-step.

June 5, 2024

by Alvin Lee

CORE

· 6,273 Views · 2 Likes

What Is Reverse ETL? Overview, Use Cases, and Key Benefits

Looking to go beyond traditional analytics? Reverse ETL is a nuanced process for businesses aiming to leverage data warehouses and other data platforms.

June 4, 2024

by Suhas Jangoan

· 6,133 Views · 2 Likes

High-Performance Storage Solutions

Pairing PostgreSQL with fast storage solutions is crucial for optimizing performance, minimizing downtime, and ensuring seamless data access.

June 3, 2024

by Sergey Platonov

· 2,551 Views · 3 Likes

An Overview of Data Pipeline Architecture

Dive into how a data pipeline helps process enormous amounts of data, key components, various architecture options, and best practices for maximum benefits.

Updated June 3, 2024

by Sreenath Devineni

CORE

· 16,543 Views · 5 Likes

Profiling Big Datasets With Apache Spark and Deequ

Struggling to maintain large dataset quality or ensure reliable data attributes in your analyses? Integrating Deequ with Spark might be the solution you need.

May 29, 2024

by Akshay Jain

· 79,626 Views · 62 Likes

Streamlining Data Warehouse Migrations From RedShift, Snowflake, and BigQuery to Apache Doris

Migrate Redshift, Snowflake, and BigQuery to Apache Doris for high-performance OLAP using tools like Apache Nifi, Talend, and Python.

May 28, 2024

by Sameer Danave

· 1,597 Views · 2 Likes

Preventing and Fixing Bad Data in Event Streams (Part 1)

In this blog, we’ll take a look at how bad data may come to be, and how we can deal with it when it comes to event streams.

May 28, 2024

by Adam Bellemare

· 4,422 Views · 3 Likes

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

Learn about ARM CPU for cost-effective and energy-efficient data streaming with Apache Kafka at the Edge and in the Cloud, including use cases.

May 27, 2024

by Kai Wähner

CORE

· 2,876 Views · 1 Like

Bridging the Gap: Exploring Innovative Paths in Data Integration

This article solely focuses on delving into understanding data integrations, associated challenges, emerging technologies, and advanced methodologies.

May 27, 2024

by Yash Mehta

· 1,305 Views · 1 Like

Apache Doris for Log and Time Series Data Analysis

NetEase has replaced Elasticsearch and InfluxDB with Apache Doris, achieving 11X query performance and saving 70% of resources.

May 25, 2024

by Shirley H.

· 2,090 Views · 1 Like

Capturing Acknowledgement in Kafka Streaming With RecordMetadata

In this article, explore RecordMetadata, and its attributes, and discover how it can be leveraged using various use cases.

May 24, 2024

by Raja Chattopadhyay

· 4,145 Views · 2 Likes

The Latest Big Data Topics