Data Engineering Resources

Data Management and Blockchain Technology

Explore how blockchain transforms data management with enhanced security, transparency, and interoperability. A game-changing synergy for future tech!

November 30, 2023

by Pamela Price

· 2,311 Views · 1 Like

Build a Philosophy Quote Generator With Vector Search and Astra DB (Part 3)

In the final installment of this series, complete the quote generator application by using an LLM to generate philosophical quotes.

November 30, 2023

by Stefano Lottini

· 2,481 Views · 1 Like

Chopping the Monolith in a Smarter Way

In this post, I offer another alternative to chop the monolith. Instead of forking the call on the client side, we fork the call on the Gateway side.

November 30, 2023

by Nicolas Fränkel

· 1,961 Views · 2 Likes

API World 2023: Bringing Together API, AI, and Secrets Security

API World 2023 was about sharing insights on best practices, accounting for all your assets, and the importance of AI and API cybersecurity in an API-driven world.

November 30, 2023

by Dwayne McDaniel

· 2,244 Views · 2 Likes

Configuring Spark-Submit

Maximize Apache Spark. Fine-tune configurations, allocate resources, and seamlessly integrate for optimal big data processing.

November 30, 2023

by Dheeraj Gupta

· 14,423 Views · 3 Likes

Best Methods To Backup and Restore Database in SQL Server

Explore methods to back up and restore the Database in the SQL Server. Get the complete solutions to Backup and Restore Database in SQL Server.

November 30, 2023

by Mithilesh Tata

CORE

· 2,949 Views · 1 Like

Decompose Legacy System Into Microservices: Part 2

In this article, follow along with a breakdown of a legacy insurance Reimbursement Calculation System into microservices in a domain-specific example.

November 29, 2023

by Somasundaram Kumarasamy

· 5,805 Views · 3 Likes

Building a Real-Time Slackbot With Generative AI

Learn how to build a cool Slackbot with Apache NiFi, LLM, Foundation Models, and streaming. We will cover model choices and integration.

November 29, 2023

by Tim Spann

CORE

· 3,546 Views · 5 Likes

IoT Cloud Computing in IoT: Benefits and Challenges Explained

In this article, you'll gain insights into the advantages, challenges, and pivotal role of cloud computing in the IoT ecosystem.

November 29, 2023

by Serhii Orlivskyi

· 5,631 Views · 2 Likes

Connecting the Dots: Unraveling IoT Standards and Protocols

MQTT facilitates efficient messaging, HTTP/HTTPS connects devices, and TLS ensures secure data exchange—integral components shaping the IoT's future.

November 29, 2023

by Animesh Patel

· 3,813 Views · 2 Likes

From Sprout to Sequoia: Orchestrating the Symphony of IoT Growth and Cloud Scalability

The IoT's evolution, from sprout to sequoia, is orchestrated by scalable cloud growth, fostering resilience, connectivity, and integration with AI.

November 29, 2023

by Animesh Patel

· 3,743 Views · 2 Likes

Demystifying Databases, Data Warehouses, Data Lakes, and Data Lake Houses

Databases for transactions, data warehouses for analytics, and data lakes and lake houses for analytics with flexibility and scalability.

November 29, 2023

by Anirudha Bhadoriya

· 4,544 Views · 9 Likes

IoT and Embedded Systems: A Brief Analysis

IoT(Internet of Things) relies on embedded systems as its foundational technology, utilizing their optimized hardware and software to process specific tasks.

November 29, 2023

by Mohit Dwivedi

· 3,232 Views · 2 Likes

Urban Connectivity 2.0: The IoT Revolution Unveiled

IoT reshapes cities, introducing smart infrastructure and venturing into quantum frontiers, ushering in a connected, innovative urban era of tomorrow.

November 29, 2023

by Animesh Patel

· 2,509 Views · 1 Like

Pipelining To Increase Throughput of Stream Processing Systems

Pipelining boosts stream processing throughput by enabling parallelism among independent threads using shared queues for coordination.

November 29, 2023

by Ankit Kumar

· 3,378 Views · 3 Likes

The Efficiency of ELT Over ETL

Understand the difference between ELT and ETL and their impact on cloud data integration.

November 29, 2023

by John Lanctot

· 2,543 Views · 2 Likes

The Role of Metadata in Data Management

Metadata is central in modern data management, vital for integration, quality, security, and key in digital transformation initiatives.

November 29, 2023

by Helen Stewart

· 3,008 Views · 2 Likes

What Is SQL Vector Database?

SQL vector databases blend the strengths of SQL and vector databases, providing a streamlined approach to storing and querying high-dimensional vectors through SQL.

November 29, 2023

by Usama Jamil

· 2,810 Views · 2 Likes

Examining and Detecting Bias in a Credit Card Defaults Dataset

This section explores bias in a credit card defaults dataset, looking at age-based discrimination leading to higher default rates among young and old borrowers.

November 29, 2023

by Serg Masis

· 2,533 Views · 2 Likes

Data-Driven Generative AI: Benefits for Data and Analytics

Generative AI is poised to revolutionize the data and analytics landscape, amplifying productivity and streamlining delivery timelines.

November 29, 2023

by Pan Singh Dhoni

· 2,347 Views · 3 Likes

The Latest Data Engineering Topics