Building Chat Apps: The Current Landscape

When it comes to building a chat application of any kind, from mobile group messaging to multiplayer in-game chat to customer support and chatbots, choosing the right platforms, frameworks, and protocols can make or break your business.

That’s because build vs. buy chat isn’t binary. The days of making a decision to do it yourself or buy from a vendor are gone.

The question now is, How much do I want to build, and how much do I want to buy?

Between open source, IaaS, PaaS, SaaS, SDKs, APIs, and microservices, businesses have never had more options for how they want to build chat products — and the spectrum of choices only continues to widen.

As cloud computing becomes more accessible and affordable, new innovative companies solve specific problems and devices become more powerful. For businesses to keep up, they must understand the vendor landscape and the benefits and challenges of each along the spectrum before they make a decision.

As a result, there are many mistakes that developers and organizations can make when choosing chat/messaging platforms for their application. In this post, we’ll discuss a number of different chat application types and look at the different platform options for powering and delivering messaging apps. We’ll also discuss challenges that can arise from making certain decisions throughout the development cycle, including scalability, time to market, and other differentiators.

Choosing a Chat Service Provider: Current Landscape

There are a wide variety of options across the build vs. buy spectrum, from open source on one end (build) to fully built-out (SaaS) solutions on the other (buy). With hundreds of options in between, all with different pros and cons, we’ll seek to give you an idea of the landscape in a simple chart.

Open-Source Protocols

The furthest on the build side are open-source protocols like WebSockets and HTTP Long Polling. These are simply protocols, which means that you manage everything to make them work. That includes spinning up your backend infrastructure, maintaining it, building new SDKs to support new devices and languages, and everything in-between.

These are great for prototyping, building small applications, or getting your hands dirty with the full stack (though, most real-time messaging services offer free versions with all the backend infrastructure included). However, gear up for some serious headaches when it’s time to scale.

Open-Source Frameworks

Open source frameworks are a smidge past pure build but still require you to maintain the infrastructure on your own. For chat use cases, open-source frameworks tend to rely on a community of developers for updating the framework and maintaining the client SDKs.

Infrastructure-as-a-Service (IaaS)

These are the big dogs – the cloud infrastructure service providers like AWS, Digital Ocean, Azure, Bluemix, and Google Cloud, that actually end up powering a lot of the PaaS, messaging solution providers, and SaaS products that we’ll talk about next.

In a nutshell, you can use open source protocols with an IaaS to launch your app. The infrastructure is taken care of, but there’s still a lot of building to do yourself.

Platform-as-a-Service (PaaS)

PaaS providers like PubNub and Firebase offer hosted solutions for building chat applications. They include not only the infrastructure but also the APIs for building chat features. Building and customizing the application requires engineering resources, as their SDKs are open, but security and maintenance of the service (the backend and the client SDKs) are handled by the PaaS.

Chat Frameworks

These framework providers are almost as close to buying as you can get but still require a fair amount of engineering. The big difference between these providers and PaaS is that they provide more of a black box approach, with less freedom to customize the APIs and infrastructure. Often they’ll provide the UI, as well.

PubNub ChatEngine offers a more open and extensible framework, while would fall closer to the black box SaaS chat solutions.

SaaS

Lastly, the farthest on the buy side of the spectrum, SaaS companies provide a fully built-out solution that requires a small amount of engineering. UI, integrations, and infrastructure are all handled by the SaaS provider. Leaders in the space include and Zendesk Chat.

Choosing Your Chat Service Provider: Questions to Ask

As with all the other parts of your critical infrastructure, the key questions still remain the same:

• Do you run your own service or do you utilize a hosted service?
• How much does it cost up front? How much will it eventually cost at scale?
• Is the hosted service reliable, secure, and scalable?
• How mission-critical is chat to my application?
• Who on my team will maintain it? Do they have the skills to make it scalable and secure?
• Where does the service store the data and who has access to it?

Choosing Your Chat Service Provider: Open-Source vs. Hosted

When it comes to software development, everyone knows that what works in the lab is not guaranteed to work in the wild. That’s because the wild has all those challenges you may not think about, or may not even know about.

When it comes to choosing the right technology to power your chat, there are a number of build and buy considerations to look at. We’ll look at the lab vs. real world, security, scalability, reliability, customization, and business reasons for selecting your stack.

Infrastructure

In going down the open-source route, you’ll choose your tool and install and orchestrate the operation of that tool.

From there, you’ll start thinking about the infrastructure side of things, like load balancing and redundant nodes, requirements for launching an app at scale. This is when you may tap an IaaS provider to handle the backend, but this will still require heavy engineering, including:

• Spinning up multiple testing, staging, production environments
• Twelve factor
• Coordinating provisioning for those multiple environments (like a Kubernetes)
• Deploying your application code to the environments
• Setting up service management, system monitoring, Ops alerting
• Creating load balancing scheme (like Nginx or HAProxy)
• Figuring out how to segment data by channels or topics (i.e. Redis pub/sub with Socket.io)
• Finding a store and forward solution for signal recovery, i.e. in-memory caching
• Implementing a method to detect which data center and port to connect a client to
• Figuring out which channels/topics to send/receive for a given client
• Deciding which platforms and languages you’ll support
• Creating universal data serialization (JSON)
• Customizing code a way to detect data uplink that works across device types
• Determining Quality of Service and level of loss; develop a data recovery scheme (or settle for “fire and forget”)
• Deciding APIs and capabilities you’ll need, then build them (i.e. presence detection)

That’s a laundry list of considerations, but if choosing the open source route, these are the things you’ll have to consider once you're outside the lab and scaling your app.

Security

For chat, security is paramount. We’re increasingly sending more confidential and mission-critical information via chat applications, from financial to chatbot commands, so ensuring that you have full control over access and encryption is imperative.

Every successful chat service provider provides different levels of security in their offering. Here are the most important features that must be included in any hosted-service provider:

• End-to-end encryption with TLS for in/outbound packets and AES for packets.
• Support fine-grained, token-based access control. Token-based access control allows you to grant and revoke access to any messaging channel.
• Complianceis key, especially for verticalized chat applications. The hosted-service provider should be certified for HIPAA (healthcare), SOC 2, GDPR (EU), Data Shield and SafeHarbor (EU/US).

For those who choose not to utilize a hosted-service provider, the following are additional security considerations that you’ll have to handle on your own:

• Purchasing a TLS certificate; distribute and manage certificate securely.
• Figuring out how to protect channels and topics (not covered by TLS).
• Build an authorization system for users.
• Consider AES and/or RSA encryption for payloads (not covered by TLS).
• Comply with legislative security policies (like SafeHarbor or HIPAA).

Scalability

For chat apps with thousands of active users chatting simultaneously and ones that continue to grow, expertise on this scale is a major challenge. Both open-source and some hosted-service providers deal with this. But when it comes to scale, hosted solutions mitigate the risk of app-breaking scalability issues far greater than open source options.

For hosted solutions, there are a couple indicators that your service of choice will scale with your app growth.

• Multiple global points of presence: Chat messages should be globally replicated, so if messages are dropped, they’ll be a backup message to be delivered. This also increases the performance of your application, as every chat user doesn’t have to connect to the same data center, especially those halfway across the earth.

• Uptime SLAs: Uptime SLAs hold hosted-service providers accountable, and they should credit you if those SLAs are not met based on the terms.

For the do-it-yourselfers, you need to consider:

• Custom build load testing service that can simulate realistic audience.
• Creating update protocol and continuously modifying your network to support new products and services.
• Pay for socket server costs, QA systems, and hot failovers.
• Ongoing Ops monitoring and additional headcount required.

Reliability

With so much competition for messaging applications and the App Store being just a click away, any issue that a user encounters can lead to them finding an alternative. Reliability is a key factor in making your app sticky. In vetting hosted service providers, here are a couple key indicators of reliability:

• Data replication for multiple points of presence and automatic failover to ensure that messages are delivered 100% of the time, and actually in real-time.
• Message “catch-up” in case of connection dropout like if a user is in a tunnel, they’ll receive the message when they come out the other side.

If using an open-source solution, you’ll have to also handle:

• Building a load distribution system.
• Identifying error messages.
• Building a log system.
• Know when faults occur and develop playbook of responses.
• Building service management (like PagerDuty).
• Developing multi-datacenter deployment.

Open-Source vs. Hosted

In looking at the major considerations, you can see that building out a real-time messaging system on your own poses a lot of risks. It is a great option for smaller chat applications, but once you begin to grow, security, reliability, and scalability challenges can add up.

Most hosted-solution providers also allow a free-forever sandbox pricing tier, where you can develop your app without paying, and once you’ve grown to a certain size, you pay as you grow. For those companies looking to move fast, and don’t want to worry about all the intricacies of networking and infrastructure, hosted-solutions are the way to go.