Apache Kafka for Telco-OTT Applications (Telecom Sector)
Next-generation architecture for the telecom sector to build Telco-OTT services, OSS/BSS integration and middleware with Apache Kafka.
Join the DZone community and get the full member experience.Join For Free
Current IT architectures in the telecom sector are not able to satisfy business needs because of their high complexity, lack of flexibility, and low level of automation. The biggest hurdle to overcome with digital transformation is to understand that it isn’t just a simple technology challenge – it covers every part of the telco business! This blog post explores next-generation architectures for the telecom sector with the Apache Kafka ecosystem to build Telco-OTT (Over the Top) services.
Apache Kafka in the Telecom Sector (OSS, BSS, Middleware, OTT)
A previous blog post covered the use cases of Event Streaming and Apache Kafka in the telecom industry: "Event Streaming and Apache Kafka in Telco Industry". Follow that post or the related "whiteboard on Youtube about Apache Kafka and the telecom sector" to learn about various use cases like the following:
Telco-OTT (Over-The-Top) - 3rd Party Telecom Services
Telco-OTT (Over-The-Top, OTT) is a particular field in the overall architecture in Telco enterprises:
- OSS (Operations Support System), as part of the telecommunication infrastructure, ensures services are working.
- BSS (Business Support System), as part of the telecommunication infrastructure, ensures that they can be provided to an actual customer
- OTT Applications use the existing telecommunication infrastructure and provide better cost and/or features and/or convenience.
An example is the easiest way to understand this better.
OTT Applications for Messaging and Chat (SMS, RCS, WhatsApp, WeChat)
Messaging applications are a great example of Telco-OTT. Easy to understand for everybody, but pretty powerful tools.
Here are some examples of messaging apps:
- SMS (short message service) by telco providers: Text format message.
- WhatsApp by Facebook: Above + group chat, gif/stickers, photos, videos, audio, location, contact information, and ‘walkie-talkie’ services.
- WeChat by Tencent: Above + payment + various other partner integrations
The telecom sector had huge revenue coming from simple text messages via SMS. No competition existed for a long time. Telcos were able to charge ~20 Cent per message. A billion-dollar business emerged.
The internet and app economy changed this massively:
People-to-People (P2P) SMS shrunk significantly. Application-to-people (A2P) is lower today, too.
The telecom sector has the following options to act when a competing service becomes successful:
- Partnership: No control over the service and potentially damage their reputation and customer relationships.
- Development of own services: No skills for building the service and too late anyway.
- Blocking OTT services: Losing revenue for traffic and customers.
Not good options, right? The best answer is to innovate early; before competitors take over.
In this case, the answer from the telco providers was RCS (rich communication services)… RCS enabled additional features to simple text, such as sending images and videos. But it came "a little bit late" and was much more expensive compared to "almost free" apps like WhatsApp and WeChat. (yes, I know, if the product is free, then you are the product - but this is another discussion...)
Next-Generation Telecom Architecture
The above example shows the main problem in the Telecommunications Industry. The telecom sector needs to completely change their strategy and overall infrastructure to stay competitive and innovative. This change is the only way to "bridge the gap with the Over-the-Top (OTT) internet providers and survive in the digital age":
Microservices Architecture for the Telecom Sector to Compete with 3rd Party OTT Applications
The telco infrastructure grew over the decades. It is inflexible, monolithic, and complex. The trend at most telco companies (I talked to in the last 12 months across the world) only sees one direction for the future: Build a flexible, open, scalable Telco architecture to process massive volumes of data in real-time.
The capability to decouple applications is where Apache Kafka shines with its distributed architecture providing a combination of messaging and storage to enable real domain-driven design (DDD):
A microservices architecture decouples applications and allows the integration between legacy and modern applications, infrastructure, and technologies. This is once again where Apache Kafka shines as scalable real-time data integration and middleware with any Telco infrastructure, using low-level interfaces such as TCP, Syslog or SNMP, business integrations with a CRM, EMS, NMS, IMS, or even mainframe offloading and replacement.
Hybrid Architectures as Key for Success in the Telco Industry
Hybrid cloud infrastructure is key to success in the telecommunications industry. The network infrastructure will always be on-premise and at the edge, while big data analytics, customer relations, and other business applications can run in a central data center or at a cloud provider:
Apache Kafka and its ecosystem provide various architecture patterns for distributed, hybrid, edge, and global deployments.
Telco architectures often combine Kafka with other Telco-specific applications and services from leading providers in the telecom sector, such as Amdocs, Ericsson or Huawei.
Disney Hotstar – OTT in Real-Time for Millions of Cricket Fans in India
Let's now take a look at a specific example for building an OTT service with requirements for real-time processing at a massive scale.
Disney+ Hotstar (known as Hotstar outside India), is an Indian over-the-top streaming service. In 2018, during the Indian Premier League, Hotstar introduced "Watch N Play", a real-time cricket prediction game in which over 33 million unique users answered over 2 billion questions, won more than 100 million rewards, built with Kafka as the backbone.
In the game, the user guesses the outcome of the next ball. If he/she guesses right before the actual result, they score points to climb up the ladder and receive rewards along the way. Supporting potentially millions of users with differing stream times & device latencies, Hotstar used topics to separate logical streams & partitions provide up to supporting 1M requests/second and more.
This is an impressive OTT service built with Apache Kafka and its ecosystem, isn't it? Check out more details in Hotstar's Kafka Summit talk "Scaling for India’s Cricket Hungry Population".
Time to Change for Traditional Players in the Telecom Sector
Deloitte has defined four scenarios for 2030. Every company in the telco industry needs to think about they want to go:
This necessary change affects any company working in the telco industry. It is time to change - no matter if you work on projects in OSS, BSS, OTT, IMS, Middleware, 3rd party services, or any other telecom domain.
Open Source MANO (OSM) for NFV and OSS Interoperability
A great example of this change in the telecom sector is the project "Open Source MANO (OSM)" for management and orchestration in OSS environments. This open-source project formed by ETSI (European Telecommunication Standards Institute) to "address challenges associated with orchestration, interoperability and performance optimization between different NFV (Network Functions Virtualization) and OSS systems":
"The unified message bus of Open Source MANO is implemented with Apache Kafka. This bus allows asynchronous communication between OSM components and enables the introduction of new modules that can be easily pluggable."
(please note that I don't like the term 'bus' because it depicts Kafka as a messaging bus even though it is an event streaming platform including messaging, storage, processing and integration capabilities)
The Evolution of the Telco Industry with Apache Kafka
The following slide deck and video recording cover the evolution of Kafka in the telco industry, including use cases, architectures, and technologies (OSS, BSS, OTT, IMS, NFV, Middleware, Mainframe, etc.):
Apache Kafka and Event Streaming for Innovation in the Telecom Sector
Current IT architectures in the telecom sector are not able to satisfy business needs because of their high complexity, lack of flexibility, and low level of automation.
Event Streaming with Apache Kafka and its ecosystem provides a scalable, reliable, and flexible infrastructure to process massive volumes in real-time. It enables real decoupling, plus powerful data integration and processing capabilities. Many telco enterprises build new infrastructures around Kafka.
This blog post focused on next-generation architecture to build Telco-OTT services. Disney+ Hotstar is an imposing example. But no matter in which part of the Telco industry you work, change and innovation are essential to staying competitive and innovative in the telecom sector.
What are your experiences with modernizing the infrastructure and applications in the telco industry? Did you or do you plan to use Apache Kafka and its ecosystem? What is your strategy? Let’s connect on LinkedIn and discuss it!
Published at DZone with permission of Kai Wähner, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.
How to Use an Anti-Corruption Layer Pattern for Improved Microservices Communication
How AI Will Change Agile Project Management
Azure Virtual Machines
A Complete Guide to Agile Software Development