Big Data Resources

Exploring Real-Time Data Ingestion Into Snowflake Using CockroachDB, Redpanda, and Kafka Connect

Explore Kafka Connect as a solution to stream changefeeds into Snowflake for greater control over how messages are delivered to Snowflake.

September 10, 2024

by Artem Ervits

CORE

· 5,510 Views · 1 Like

Accelerate Your Journey to a Modern Data Platform Using Coalesce

This article will identify key challenges organizations face today in managing data platforms, and explore how advanced ETL tools can address these challenges.

September 10, 2024

by Asia Banu Shaik

· 5,086 Views · 5 Likes

Data Storage Formats for Big Data Analytics: Performance and Cost Implications of Parquet, Avro, and ORC

This article compares the performance and cost efficiency of three storage formats Parquet, Avro, and ORC on Google Cloud Platform.

September 9, 2024

by Rahul Sarabu

· 5,863 Views · 4 Likes

Principles of Modern Data Infrastructure

Explore principles of modern data infrastructure such as scalability, high availability, speed, security, maintainability, efficiency, and developer experience.

September 6, 2024

by Joe Zhou

· 5,170 Views · 1 Like

Keeping Two Multi-Master Databases Aligned With a Vector Clock

In this article, learn about an experience in keeping two different databases aligned with two different technologies by using an application-level solution.

September 5, 2024

by Claudio Guidi

CORE

· 6,176 Views · 4 Likes

Setting Up a Data Warehouse for Starlight: A Comprehensive Guide

Learn architectural considerations, essential tools, and technologies, and see sample code snippets to illustrate key steps of a data warehouse setup.

September 5, 2024

by Harsh Daiya

CORE

· 3,920 Views · 1 Like

How to Conduct Effective Data Security Audits for Big Data Systems

Learn key strategies for conducting thorough data security audits in big data systems to safeguard sensitive information.

September 4, 2024

by Devin Partida

· 5,111 Views · 1 Like

MLOps: How to Build a Toolkit to Boost AI Project Performance

AI projects could end up among the 90% that fail due to common implementation pitfalls. Here, learn how to change the game with the right MLOps tools.

September 3, 2024

by Alexander Simonov

· 6,124 Views · 2 Likes

Open Standards for Data Lineage: OpenLineage for Batch and Streaming

Explores trends and efforts to provide an open standard with OpenLineage, and how data governance solutions help fulfill enterprise-wide data governance needs.

September 3, 2024

by Kai Wähner

CORE

· 6,021 Views · 1 Like

Confusion Matrix vs. ROC Curve: When to Use Which for Model Evaluation

The Confusion Matrix and the ROC Curve evaluate model performance in machine learning and data science. Compare and learn when to use each in model evaluation.

September 3, 2024

by Fizza Jatniwala

· 5,294 Views · 1 Like

Telemetry Pipelines Workshop: Integrating Fluent Bit With OpenTelemetry, Part 2

Take a look at integrating Fluent Bit with OpenTelemetry for use cases where this connectivity in your telemetry data infrastructure is essential.

September 2, 2024

by Eric D. Schabell

CORE

· 6,362 Views · 1 Like

Building Product to Learn AI, Part 3: Taste Testing and Evaluating

Before our AI meal planner is complete, one crucial step remains: a taste test. This is where a robust evaluation framework becomes your most valuable tool.

September 2, 2024

by Indrajit Bhattacharya

· 2,867 Views · 2 Likes

Best Practices for Salesforce Data Management and Security

Are you doing enough to ensure data security while adhering to data hygiene best practices? Here, learn Salesforce data management and security best practices.

September 2, 2024

by Ilya Dudkin

CORE

· 3,277 Views · 1 Like

High Fidelity Data: Balancing Privacy and Usage

Explore High Fidelity (HiFi) Data features such as visual, population, statistical, and ownership integrity, and its use to balance privacy and usability.

August 29, 2024

by Zilong Tang

· 5,800 Views · 4 Likes

Effortless Concurrency: Leveraging the Actor Model in Financial Transaction Systems

Learn challenges of financial transaction systems and implement robust mechanisms to handle concurrency effectively to ensure system integrity and reliability.

August 28, 2024

by Nikita Melnikov

· 30,235 Views · 6 Likes

Maximizing Enterprise Data: Unleashing the Productive Power of AI With the Right Approach

Learn how organizations can unlock the full potential of their AI initiatives by considering critical elements such as infrastructure, cost, and more.

August 26, 2024

by Pan Singh Dhoni

· 4,749 Views · 1 Like

Methodcentipede

See an example of an antipattern that can lead to difficulties in maintaining and testing code and approaches that allow you to structure your work in a preventative way.

August 26, 2024

by Anton Belyaev

· 8,849 Views · 3 Likes

The Role of Data Governance in Data Strategy: Part 3

Following our look into the enhancement of data in an org, explore Data Subject Access Rights (DSAR) and their correlation to individual rights in real-time.

August 23, 2024

by Satish Gaddipati

· 4,635 Views · 3 Likes

Integrating Apache Kafka in KRaft Mode With RisingWave for Event Stream Processing

Install and configure the latest version of Apache Kafka on a single-node cluster running on Ubuntu-22.04, and subsequently integrate it with RisingWave.

August 23, 2024

by Gautam Goswami

CORE

· 4,811 Views · 2 Likes

Data Pipeline Techniques in Action

Take a deep dive into the architectural concepts of data pipelines along with a hands-on tutorial for implementation, demonstrating the concepts in action.

August 21, 2024

by Sohil Shah

· 6,653 Views · 2 Likes

The Latest Big Data Topics