Big Data Resources

Automate Private Azure Databricks Unity Catalog Creation

This article provides detailed steps to automate a private Databricks Unity Catalog creation in Azure subscription.

October 21, 2024

by Soumya Barman

· 3,371 Views · 1 Like

Tech Trends 2024: Highlights on the Current Tech Industry From a Developer

The tech landscape is changing fast. Get the pulse on the latest topics and trends in technology from one developer's perspective.

October 21, 2024

by Jennifer Reif

CORE

· 3,536 Views · 1 Like

The Battle of Data: Statistics vs Machine Learning

Compare statistics and machine learning, discussing their foundations, methods, applications, and differences in analyzing data for insights and predictions.

October 14, 2024

by Vasanthi Govindaraj

· 4,795 Views · 8 Likes

Hello, K.AI: How I Trained a Chatbot of Myself Without Coding

Follow a journey without coding to train a personal chatbot that can be used to learn in a conversational pace format about data streaming.

October 14, 2024

by Kai Wähner

CORE

· 2,163 Views · 1 Like

Optimizing IoT Performance in Industrial Environments

Read about the hardware and software optimizations to implement to maximize IoT performance in industrial environments.

October 9, 2024

by Zac Amos

· 4,955 Views · 2 Likes

Enhancing Performance With Data Modeling: Techniques and Best Practices for Optimization in Snowflake

Explore advanced data modeling techniques, focusing on star schema, snowflake schema, and hybrid approaches.

October 8, 2024

by Harshavardhan Yedla

· 4,732 Views · 3 Likes

The Shift Left Architecture: From Batch and Lakehouse to Data Streaming

Explore the Shift Left Architecture using data streaming (Kafka/Flink) that enables data products for DWH, data lake, and lakehouses like Snowflake/Databricks.

October 8, 2024

by Kai Wähner

CORE

· 3,522 Views · 1 Like

Transforming Manufacturing: AI-Driven Quality Control and Predictive Maintenance Using Cloud and IoT

In modern manufacturing, the key to sustaining product reliability, compliance, and efficiency lies in robust quality control and advanced maintenance management.

October 2, 2024

by vuppulapati Chandra Sekhar Naidu

· 3,550 Views · 1 Like

Top 6 Cybersecurity Threat Detection Use Cases: How AI/ML Can Help Detect Advanced and Emerging Threats

Learn how to get started to design and build AI/ML solutions that are capable of revolutionizing the cybersecurity threat detection landscape.

October 2, 2024

by Mayank Singhi

· 3,351 Views · 2 Likes

Leveraging Artificial Intelligence for Software Testing to Test Healthcare Applications

Learn how leveraging AI-powered software testing for healthcare domains can make a significant difference in the quality, efficiency, and reliability of care.

October 1, 2024

by Vikram Singh

· 3,491 Views · 5 Likes

What Is Lambda Architecture? Ultimate Guide to Getting Started

Discover the basics of lambda architecture in our comprehensive guide. Explore its components, workflow, applications, and best practices.

Updated October 1, 2024

by Michael Bogan

CORE

· 15,034 Views · 11 Likes

5 Best Practices for Data Warehousing

Getting started in data warehousing is a major undertaking, so it's important to consider a few best practices before beginning.

Updated October 1, 2024

by Zac Amos

· 6,815 Views · 2 Likes

Stream Processing in the Serverless World

Learn how real-time data processing enables businesses to gain actionable insights without the overhead of managing infrastructure.

September 30, 2024

by Rajesh Pandey

· 4,205 Views · 3 Likes

Assigning Pods to Nodes Using Affinity Rules

Affinity and anti-affinity are Kubernetes features to help you manage the schedule of your applications. Explore two scenarios where to use these features.

September 27, 2024

by Rafael Natali

· 4,789 Views · 4 Likes

Is Spring AI Strong Enough for AI?

Explore Spring's capabilities within the AI domain, its potential integration with AI libraries, and its ability to effectively manage AI workflows.

September 27, 2024

by Reza Ganji

CORE

· 12,662 Views · 13 Likes

Integrating Data Engineering Into Artificial Intelligence

Importance of tight coupling between data engineering and artificial intelligence for data engineers and data scientists and best practices.

September 26, 2024

by Akshay Agarwal

· 2,899 Views · 3 Likes

Using Flink, Kafka, and NiFi for Real-Time Airport Arrivals and Departures

Learn to build a streaming application using the best of NiFi, Kafka, and Flink for event-driven apps. OpenSky networks rest feeds provide all the data.

September 26, 2024

by Timothy Spann

· 3,830 Views · 5 Likes

Automating Data Pipelines With Snowflake: Leveraging DBT and Airflow Orchestration Frameworks for ETL/ELT Processes

This article explores the automation of data pipelines using Snowflake, dbt, and Airflow, detailing best practices for efficient data processing and orchestration.

September 25, 2024

by Harshavardhan Yedla

· 4,046 Views · 2 Likes

Feature Engineering Transforming Predictive Models

Delve into the transformative power of feature engineering in applied machine learning, and learn how carefully crafted features can elevate your models.

September 25, 2024

by Sundeep Goud Katta

· 3,170 Views · 3 Likes

What Is a Data Pipeline?

In this post, we examine data pipelines, explore their benefits, compare them to other data processes, and discuss various implementation methods.

Updated September 24, 2024

by Garrett Alley

· 50,967 Views · 18 Likes

The Latest Big Data Topics