Data Engineering Resources

Upgrading a Database Project to Python 3.12

Upgrading to Python versions can be tricky, particularly for Python 3.12. This article describes an experience for a database-oriented project.

April 3, 2024

by Val Huber

CORE

· 2,248 Views · 2 Likes

Exploring Debugging in Apache Airflow: Strategies and Solutions

This article delves into effective debugging techniques in Apache Airflow, focusing on troubleshooting stuck tasks and other common issues.

April 3, 2024

by kalpesh barde

· 2,081 Views · 1 Like

Fear and Loath Building Data Lineage

Discover how intuitive product designs reshape the approach towards understanding complex data flows and enhancing user experience.

April 3, 2024

by Yuliia Tkachova

· 915 Views · 2 Likes

Next-Gen Data Protection: Navigating Data Security Challenges in 2024

Underscores the critical need for advanced security measures, innovative technologies, and comprehensive strategies to combat cyber threats and safeguard digital assets.

April 3, 2024

by Alex Tray

· 2,069 Views · 1 Like

Rapid Expansion of Generative AI: A Call for Responsible Product Management

Responsible AI product management ensures ethical, safe, and fair AI use, focusing on integrating key principles and safety features.

April 3, 2024

by Chetan Zawar

· 1,091 Views · 2 Likes

AI-Powered DDR: Supercharging Your Threat Detection Capabilities

In the relentless battle against cyber threats, time is of the essence. Integrating AI into DDR solutions marks a turning point in the fight against cybercrime.

April 3, 2024

by Anastasios Arampatzis

· 1,325 Views · 1 Like

AI Frameworks for Software Engineers: TensorFlow (Part 1)

TensorFlow, a key AI framework, enables the building, training, and deploying of versatile AI models, illustrated through a simple example.

April 3, 2024

by Amardeep Singh

· 5,741 Views · 7 Likes

Ethics in the Age of AI: The Human and Moral Impact of AI

Generative AI and LLMs have ever-expanding use cases, but they also have the potential for misuse, raising fundamental questions regarding AI's human and ethical impact.

April 3, 2024

by Cate Lawrence

CORE

· 4,939 Views · 3 Likes

Optimize Data Warehouse Migration Efforts by Factor of 50

This paper outlines the challenges encountered during the migration of a large data warehouse and how these challenges were tackled through innovative solutions.

April 3, 2024

by Vaibhav Dantale

· 17,156 Views · 3 Likes

Understanding Firewalls: Your First Line of Cyber Defense

In this article, we'll delve into the significance of firewalls, their types, how they function, and best practices for implementing them effectively.

April 3, 2024

by Aditya Bhuyan

· 3,617 Views · 2 Likes

Efficient Ticket Management Using LangChain and Large Language Models

LangChain agents, powered by ChatGPT, revolutionize JIRA ticket creation for seamless cross-team collaboration, boosting efficiency and productivity.

April 3, 2024

by Rupali Kaushik

· 8,765 Views · 5 Likes

Building Resilient Cybersecurity Into Supply Chain Operations: A Technical Approach

Adding cybersecurity measures into each facet of the supply chain safeguards operations and sensitive data, giving a competitive edge in the digital marketplace.

April 3, 2024

by Akanksha Pathak

CORE

· 4,134 Views · 3 Likes

Automating Operational Efficiency: Integrating AI Insights From Amazon SageMaker Into Business Workflows

Integrating AI within AWS RDS MySQL for managing terabytes of flight data on a weekly basis involves leveraging AWS's vast ecosystem of AI and data services.

April 2, 2024

by Vijay Panwar

CORE

· 1,643 Views · 1 Like

Data Lake vs. Data Warehouse

Data lakes offer flexibility with raw data; data warehouses provide structured data for quick insights. Each has its own benefits and trade-offs.

April 2, 2024

by Pier-Jean MALANDRINO

CORE

· 1,606 Views · 3 Likes

Node-RED Unleashed: Transforming Industrial IoT Development and Industry Collaboration With Hitachi

Hear from Hitachi's Kazuhito Yokoi to find out more about Node-RED and why it is becoming so popular in Industrial IoT applications.

April 2, 2024

by Jesse Casman

CORE

· 11,245 Views · 7 Likes

Demystifying Event Storming: Design Level, Identifying Bounded Contexts (Part 4)

Learn how event storming aids in identifying and defining bounded contexts, explore insights on different types, and discuss Conway's Law and external systems' influence.

April 2, 2024

by Alireza Rahmani Khalili

CORE

· 4,795 Views · 3 Likes

DSL Validations: Operators

Now that we have seen how the DSL validates individual properties, the next step is to combine individual property validations into larger, more complex conditionals.

April 2, 2024

by Scott Sosna

CORE

· 1,782 Views · 1 Like

OpenAI Evals Using Phoenix

OpenAI Evals are used for evaluating LLM models and finding accuracy, which helps to compare the custom models and figure out how best your custom model performs.

April 2, 2024

by Somanath Balakrishnan

CORE

· 2,121 Views · 1 Like

Understanding Prompt Injection and Other Risks of Generative AI

By prioritizing security, organizations can enhance trust, resilience, and reliability in their cloud and AI environments.

April 2, 2024

by Vidyasagar (Sarath Chandra) Machupalli FBCS

CORE

· 10,719 Views · 5 Likes

The Evolution of Conversational AI: From Chatbots to Coherent Conversations With GenAI and LLMs

Conversational AI has evolved, transitioning from simplistic rule- or FAQ-based systems to advanced virtual assistants capable of human-like dialogue.

April 2, 2024

by Pratik Prakash

CORE

· 6,596 Views · 3 Likes

The Latest Data Engineering Topics