Kai Wähner

CORE

Technology Evangelist at Confluent

Effeltrich, DE

Joined Jun 2010

https://www.kai-waehner.de

About

Kai Waehner works as Technology Evangelist at Confluent. Kai’s main area of expertise lies within the fields of Big Data Analytics, Machine Learning / Deep Learning, Messaging, Integration, Microservices, Internet of Things, Stream Processing and Blockchain. He is regular speaker at international conferences such as JavaOne, O’Reilly Software Architecture or ApacheCon, writes articles for professional journals, and shares his experiences with new technologies on his blog (www.kai-waehner.de/blog). Contact and references: kontakt@kai-waehner.de / @KaiWaehner / www.kai-waehner.de

Stats

Reputation:	6739
Pageviews:	3.4M
Articles:	137
Comments:	5

Expertise

Data

IoT

Languages

Articles
Comments

Articles

Deployment Strategies for Apache Kafka Cluster Types

Multiple Kafka clusters enable hybrid integration, aggregation, migration, and disaster recovery across edge, data center, and multi-cloud environments.

November 19, 2024

· 1,683 Views · 1 Like

Apache Iceberg: The Open Table Format for Lakehouses and Data Streaming

This article explores the table format wars of Apache Iceberg, Hudi, Delta Lake and XTable; and the product strategy of Snowflake, Databricks, Confluent, AWS, and Google.

November 12, 2024

· 3,593 Views · 1 Like

Digitalization of Airport and Airlines With IoT and Data Streaming Using Kafka and Flink

Take an in-depth look at IoT and data streaming using Kafka and Flink at airports such as Schiphol AMS and airlines like Lufthansa and Cathay.

November 4, 2024

· 9,749 Views · 2 Likes

Hello, K.AI: How I Trained a Chatbot of Myself Without Coding

Follow a journey without coding to train a personal chatbot that can be used to learn in a conversational pace format about data streaming.

October 14, 2024

· 2,085 Views · 1 Like

The Shift Left Architecture: From Batch and Lakehouse to Data Streaming

Explore the Shift Left Architecture using data streaming (Kafka/Flink) that enables data products for DWH, data lake, and lakehouses like Snowflake/Databricks.

October 8, 2024

· 3,397 Views · 1 Like

Real-Time GenAI With RAG Using Apache Kafka and Flink to Prevent Hallucinations

Learn about context-specific real-time Generative AI (GenAI) with Retrieval Augmentation Generation (RAG) using Kafka and Flink to prevent hallucinations.

September 11, 2024

· 3,288 Views · 1 Like

Open Standards for Data Lineage: OpenLineage for Batch and Streaming

Explores trends and efforts to provide an open standard with OpenLineage, and how data governance solutions help fulfill enterprise-wide data governance needs.

September 3, 2024

· 5,727 Views · 1 Like

Apache Kafka + Flink + Snowflake: Cost-Efficient Analytics and Data Governance

Explore shift-left architecture with Snowflake, Apache Kafka, Flink, and Iceberg for cost-efficient analytics and data governance in the cloud.

August 10, 2024

· 5,381 Views · 1 Like

Snowflake Integration Patterns: Apache Kafka vs. Zero ETL and Reverse ETL

Integration patterns for cloud-native Snowflake data warehouse: Batch data, Zero ETL, Reverse ETL, Real-time data ingestion with Apache Kafka.

July 24, 2024

· 5,572 Views · 2 Likes

When Not to Choose Google Apache Kafka for BigQuery

When not to choose Google Apache Kafka for BigQuery — a review, comparison with Confluent Cloud, and data streaming landscape outlook.

July 16, 2024

· 4,522 Views · 1 Like

When Not To Use Apache Kafka (Lightboard Video)

When not to use Apache Kafka, do's and don't's; no matter if you use open source, Confluent, Amazon MSK, Event Hubs, Redpanda, Warpstream, et al.

June 25, 2024

· 2,332 Views · 1 Like

The Past, Present, and Future of Stream Processing

Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI, and Apache Iceberg.

June 17, 2024

· 8,289 Views · 7 Likes

JavaScript, Node.js, and Apache Kafka for Full-Stack Data Streaming

Explore JavaScript, Node.js, and Apache Kafka for full-stack development for data streaming with open-source clients.

June 10, 2024

· 6,646 Views · 2 Likes

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

Learn about ARM CPU for cost-effective and energy-efficient data streaming with Apache Kafka at the Edge and in the Cloud, including use cases.

May 27, 2024

· 2,035 Views · 1 Like

GenAI Demo With Kafka, Flink, LangChain and OpenAI

Explore the architecture and demo for open-source data streaming using Apache Kafka and Flink with Python, LangChain, and OpenAI LLM APIs in the cloud.

May 17, 2024

· 2,479 Views · 1 Like

The Data Streaming Landscape 2024

This article presents a comparison of open-source Apache Kafka and Flink stream processing products, cloud, competition, and market trends.

March 20, 2024

· 8,452 Views · 2 Likes

MQTT Market Trends for 2024: Cloud, Unified Namespace, Sparkplug, Kafka Integration

Market trends for MQTT: Data Governance with Unified Namespace and Sparkplug B, OPC-UA Debates, and Integration with Apache Kafka for OT/IT.

March 13, 2024

· 2,898 Views · 1 Like

Why Tiered Storage for Apache Kafka Is a Big Thing

This blog post explores the architecture, use cases, benefits, and a case study for storing Petabytes of data in the Kafka commit log.

February 26, 2024

· 1,704 Views · 1 Like

Top 5 Trends for Data Streaming With Kafka and Flink in 2024

Top 5 Trends for Data Streaming with Apache Kafka and Flink: data sharing, data contracts, multi-cloud, serverless stream processing, GenAI.

February 19, 2024

· 4,214 Views · 2 Likes

The State of Data Streaming for Healthcare With Kafka and Flink

Data Streaming with Apache Kafka and Apache Flink enables IT modernization and innovation at healthcare companies like Humana, BHG, and Recursion.

February 9, 2024

· 3,928 Views · 1 Like

Apache Kafka + Vector Database + LLM = Real-Time GenAI

How Apache Kafka, Flink, and Vector Databases with semantic search make an LLM and GenAI reliable with real-time context.

January 31, 2024

· 2,621 Views · 1 Like

The State of Data Streaming With Apache Kafka and Flink in the Gaming Industry

This article covers the architectures, use cases, and case studies for data streaming with Kafka and Flink in the gaming industry, including Kakao Games, Blizzard, and MPL.

January 26, 2024

· 4,588 Views · 1 Like

How Apache Kafka Helps Dish Wireless Building Cloud-Native 5G Telco Infrastructure

Apache Kafka helps Dish Wireless as data fabric in its cloud-native 5G infrastructure for real-time use cases in retail, logistics, IoT, and so on.

January 19, 2024

· 4,331 Views · 2 Likes

Policy Enforcement and Data Quality for Apache Kafka With Schema Registry

Data governance with policy enforcement and data contracts for good data quality in Apache Kafka using enhanced Schema Registry APIs.

January 4, 2024

· 4,525 Views · 1 Like

How Michelin Improves Aftermarket Sales and Customer Service With Apache Kafka

How Michelin improves aftermarket sales and customer service in the automotive industry with business process choreography using Apache Kafka.

December 22, 2023

· 3,538 Views · 1 Like

How Lufthansa Uses Apache Kafka for Data Integration and Machine Learning

The airline Lufthansa uses Apache Kafka as cloud-native middleware for data integration and as data fabric for analytics and machine learning.

December 19, 2023

· 4,199 Views · 2 Likes

The State of Data Streaming for Insurance in 2023

Explore data streaming powered by Apache Kafka in the insurance industry, including use cases and cloud architectures of Allianz, Generali, and more.

December 8, 2023

· 2,387 Views · 1 Like

Real-Time Advertising With Apache Kafka and Flink

Real-time advertising platform built with Apache Kafka and Apache Flink showing case studies of Pinterest, Uber, Unity, Buzzwil, Reddit, etc.

December 8, 2023

· 5,234 Views · 1 Like

Modernizing SCADA Systems and OT/IT Integration With Apache Kafka

Explore the modernization of SCADA Systems and hybrid cloud OT/IT infrastructure with data streaming powered by Apache Kafka.

November 16, 2023

· 3,290 Views · 4 Likes

The State of Data Streaming for Energy and Utilities in 2023

Use cases, architectures, and case studies for Apache Kafka in the energy and utilities industry, including IoT, AI, blockchain, and cybersecurity.

November 3, 2023

· 4,032 Views · 2 Likes

Comments

Spring Integration and Apache Camel

Jan 11, 2012 · Biju Kunjummen

I just wrote an article, which explains my experiences with Apache Camel and its alternatives Spring Integration and Mule ESB:

Spoilt for Choice: Which Integration Framework to use - Spring Integration, Mule ESB or Apache Camel?

Best regards,
Kai Wähner (Twitter: @KaiWaehner)

Pros and Cons – When to use a Portal and Portlets instead of just Java Web-Frameworks

Oct 14, 2011 · Kai Wähner

Thanks for your comments.

@André: Well, you are right: You should not consider only product costs (they are not much cheaper for Liferay than for Oracle), but the whole development cycle. Maybe my article was a little bit unclear about this!

I see, you have a lot more experience with Portlets than me. We did not go any further, because we decided that a Portal / Portlets are not reasonable for our project.

Regarding the Oracle stack: We only evaluated the Portal which Oracle offered us combined with its Oracle SOA Suite (WebCenter). I did not make the choice, I just had to evaluate it...

@Erron: As already mentioned, I do not have much experience with Portlets. We decided that it is too much extra effort without creating a prototype or so :-) Thus, I cannot tell you anything about the GWT Portlet plugin. Maybe someone else can?

Open Source Integration With Apache Camel and How Fuse IDE Can Help

May 19, 2011 · mitchp

Hey Jonathan, this article is a good overview about Apache Camel. I can confirm that all promises about easier integration are true (because you always use the same DSL). I already tried out the BETA of Fuse IDE and wonder if there are plans for supporting Java DSL in the future? Yes, I am one of the guys, who does not like too much XML configuration :-) Best regards, Kai Wähner (Twitter: @KaiWaehner)

Creating Application using Spring Roo and Deploying on Google App Engine

Apr 10, 2011 · Shekhar Gulati

Hey Shekhar,

good article. But did you also realize some relationships between entities? I think "OneToMany" and so on do not work with GAE?

Best regards,

Kai Wähner (Twitter: @KaiWaehner)

Apache Camel: Integration Nirvana

Jan 04, 2011 · Jonathan Anstey

Very nice introduction to Apache Camel! Thank you.