The State of Data Streaming for Telco

This blog post explores the state of data streaming for the telco industry in 2023. The evolution of telco infrastructure, customer services, and new business models requires real-time end-to-end visibility, fancy mobile apps, and integration with pioneering technologies like 5G for low latency or augmented reality for innovation. Data streaming allows integrating and correlating data in real-time at any scale to improve most telco workloads.

I look at trends in the telecommunications sector to explore how data streaming helps as a business enabler, including customer stories from Dish Network, British Telecom, Globe Telecom, Swisscom, and more. A complete slide deck and on-demand video recording are included.

General Trends in the Telco Industry

The Telco industry is fundamental for growth and innovation across all industries.

The global spending on telecom services is expected to reach 1.595 trillion U.S. dollars by 2024 (Source: Statista, Jul 2022). 

Cloud-native infrastructure and digitalization of business processes are critical enablers. 5G network capabilities and telco marketplaces enable entirely new business models.

5G Enables New Business Models

Presentation of Amdocs / Mavenir:

A report from McKinsey and Company says, “74 percent of customers have a positive or neutral feeling about their operators offering different speeds to mobile users with different needs”. The potential for increasing the revenue per user (ARPU) with 5G use cases is enormous for telcos:

Telco Marketplace

Many companies across industries are trying to build a marketplace these days. But the telecom sector might shine here because of its interface between infrastructure, B2B, partners, and end users for sales and marketing.

tmforum has a few good arguments for why communication service providers (CSP) should build a marketplace for B2C and B2B2X:

• Operating the marketplace keeps CSP in control of the relationship with customers
• A marketplace is a great sales channel for additional revenue
• Operating the marketplace helps CSPs monetize third-party (over-the-top) content
• The only other option is to be relegated to connectivity provider
• Enterprise customers have decided this is their preferred method of engagement
• CPSs can take a cut of all sales
• Participating in a marketplace prevents any one company from owning the customer

Data Streaming in the Telco Industry

Adopting trends like network monitoring, personalized sales, and cybersecurity is only possible if enterprises in the telco industry can provide and correlate information at the right time in the proper context. Real-time, which means using the information in milliseconds, seconds, or minutes, is almost always better than processing data later (whatever later means):

Data streaming combines the power of real-time messaging at any scale with storage for true decoupling, data integration, and data correlation capabilities.

"Use Cases for Apache Kafka in Telco" is a good article for starting with an industry-specific point of view on data streaming. "Apache Kafka for Telco-OTT and Media Applications" explores over-the-top B2B scenarios.

Data streaming with the Apache Kafka ecosystem and cloud services are used throughout the supply chain of the telco industry. 

From Telco to TechCo: Next-Generation Architecture

Deloitte describes the target architecture for telcos very well:

Data streaming provides these characteristics: Open, scalable, reliable, and real-time. This unique combination of capabilities made Apache Kafka so successful and widely adopted.

Kafka decouples applications and is the perfect technology for microservices across a telco's enterprise architecture. Deloitte's diagram shows this transition across the entire telecom sector:

This is a massive shift for telcos:

• From purpose-built hardware to generic hardware and elastic scale
• From monoliths to decoupled, independent services

Digitalization with modern concepts helps a lot in designing the future of telcos.

Open Data Architecture (ODA)

tmforum describes Open Digital Architecture (ODA) as follows:

"Open Digital Architecture is a standardized cloud-native enterprise architecture blueprint for all elements of the industry from Communication Service Providers (CSPs) through vendors to system integrators. It accelerates the delivery of next-gen connectivity and beyond — unlocking agility, removing barriers to partnering, and accelerating concept-to-cash.

ODA replaces traditional operations and business support systems (OSS/BSS) with a new approach to building software for the telecoms industry, opening a market for standardized, cloud-native software components and enabling communication service providers and suppliers to invest in IT for new and differentiated services instead of maintenance and integration."

If you look at the architecture trends and customer stories for data streaming in the next section, you realize that real-time data integration and processing at scale are required to provide the most modern use cases in the telecommunications industry.

Architecture Trends for Data Streaming

The telco industry applies various trends for enterprise architectures for cost, flexibility, security, and latency reasons. The three major topics I see these days at customers are:

• Hybrid architectures with synchronization between edge and cloud in real-time
• End-to-end network and infrastructure monitoring across multiple layers
• Proactive service management and context-specific customer interactions

Let's look deeper into some enterprise architectures that leverage data streaming for telco use cases.

Hybrid 5G Architecture With Data Streaming

Most telcos have a cloud-first strategy to set up modern infrastructure for network monitoring, sales and marketing, loyalty, innovative new OTT services, etc. However, edge computing gets more relevant for use cases like pre-processing for cost reduction, innovative location-based 5G services, and other real-time analytics scenarios:

Learn about architecture patterns for Apache Kafka that may require multi-cluster solutions and see real-world examples with their specific requirements and trade-offs. That blog explores scenarios such as disaster recovery, aggregation for analytics, cloud migration, mission-critical stretched deployments, and global Kafka.

Edge deployments for data streaming are their own challenges. In separate blog posts, I covered use cases for Kafka at the edge and provided an infrastructure checklist for edge data streaming.

End-To-End Network and Infrastructure Monitoring

Data streaming enables unifying telemetry data from various sources such as Syslog, TCP, files, REST, and other proprietary application interfaces:

End-to-end visibility into the telco networks allows massive cost reductions. And, as a bonus, a better customer experience. For instance, proactive service management tells customers about a network outage:

Context-Specific Sales and Digital Lifestyle Services

Customers expect a great customer experience across devices (like a web browser or mobile app) and human interactions (e.g., in a telco store). Data streaming enables a context-specific omnichannel sales experience by correlating real-time and historical data at the right time in the proper context:

"Omnichannel Retail and Customer 360 in Real Time with Apache Kafka" goes into more detail. But one thing is clear: Most innovative use cases require both historical and real-time data. In summary, correlating historical and real-time information is possible with data streaming out-of-the-box because of the underlying append-only commit log and replayability of events. A cloud-native Tiered Storage Kafka infrastructure to separate compute from storage makes such an enterprise architecture more scalable and cost-efficient.

The article "Fraud Detection with Apache Kafka, KSQL, and Apache Flink" explores stream processing for real-time analytics in more detail, shows an example with embedded machine learning, and covers several real-world case studies.

New Customer Stories for Data Streaming in the Telco Industry

So much innovation is happening in the telecom sector. Automation and digitalization change how telcos monitor networks, build customer relationships, and create completely new business models.

Most telecommunication service providers use a cloud-first approach to improve time-to-market, increase flexibility, and focus on business logic instead of operating IT infrastructure. Elastic scalability gets even more critical with all the growing networks and 5G workloads.

Here are a few customer stories from worldwide telecom companies:

• Dish Network: Cloud-native 5G Network with Kafka as the central communications hub between all the infrastructure interfaces and IT applications. The standalone 5G infrastructure, in conjunction with data streaming, enables new business models for customers across all industries, like retail, automotive, or energy sector.
• Verizon: MEC use cases for low-latency 5G stream processing, such as autonomous drone-in-a-box-based monitoring and inspection solutions or vehicle-to-Everything (V2X).
• Swisscom: Network monitoring and incident management with real-time data at scale to inform customers about outages, root cause analysis, and much more. The solution relies on Apache Kafka and Apache Druid for real-time analytics use cases.
• British Telecom (BT): Hybrid multi-cloud data streaming architecture for proactive service management. BT extracts more value from its data and prioritizes real-time information and better customer experiences.
• Globe Telecom: Industrialization of event streaming for various use cases. Two examples: Digital personalized rewards points based on customer purchases. Airtime loans are made easier to operationalize (vs. batch, where top-up cash is already spent again).

Resources To Learn More

This blog post is just the starting point. Learn more about data streaming in the telco industry in the following on-demand webinar recording, the related slide deck, and further resources, including pretty cool lightboard videos about use cases.

On-Demand Video Recording

The video recording explores the telecom industry's trends and architectures for data streaming. The primary focus is the data streaming case studies. Check out our on-demand recording.

Slides

If you prefer learning from slides, check out the deck used for the above recording:

Slides: The Data of Apache Kafka for Telco in 2023

Case Studies and Lightboard Videos for Data Streaming in Telco

The state of data streaming for Telco in 2023 is fascinating. New use cases and case studies come up every month. This includes better data governance across the entire organization, real-time data collection and processing data from network infrastructure and mobile apps, data sharing and B2B partnerships with OTT players for new business models, and many more scenarios.

We recorded lightboard videos showing the value of data streaming simply and effectively. These five-minute videos explore the business value of data streaming, related architectures, and customer stories. Stay tuned; I will update the links in the next few weeks and publish a separate blog post for each story and lightboard video.

And this is just the beginning. Every month, we will talk about the status of data streaming in a different industry. Manufacturing was the first. Financial services second, then retail, telcos, gaming, and so on.