How To Use Kubernetes in AI Projects

Introduction

The Cloud Native Survey, which polls Technology/Software organizations, reports that the use of containers in production has increased by 84% from the previous year, up to 92%. The use of Kubernetes is up 78% from the previous year, reaching 83%.

Kubernetes’ distributed architecture and scalability pair well with Machine Learning and Artificial Intelligence. As these technologies continue to mature, 2021 is the year to watch for growth in this space.

Companies should keep in mind that tools support business goals, and are not an end to themselves. 2020 was a year in which nearly every business was forced to reckon with unexpected changes. Kubernetes has the capability to accelerate the application development services through solutions built with a cloud-native ecosystem, while at the same time allowing for malleable use of applications and data so that a company can succeed through the modernization of its platforms and apps.

Below there are two real-world use cases of how Kubernetes helped optimize AI project's performance and costs.

Kubernetes Use Case #1

Data Engineering for AI-based Demand Forecasting System

Project Description

A POS software and venue management system adopted by a lot of bars and restaurants. Over time, a huge amount of historical POS sales data accumulated, and MobiDev’s tech experts came up with an idea to apply artificial intelligence (AI) algorithms to the data, to find patterns in the previous sales and make predictions for the next period for each venue. An AI-based demand forecasting system was developed, as an independent module integrated into the system.

The Problems to Solve

As AI computations required significant resources, initially, virtual machines in the AWS EMR cloud service were used. The more venues that adopted the system, the more expensive the infrastructure became. The AI module’s high CPU load occurred overnight when machine learning algorithms were processing the daily sales data and were idle during the day. In order to decrease infrastructure costs, manual management of computing resources was implemented with the help of Docker Swarm.

At the MVP development stage, it didn’t make sense to use Kubernetes as it would require a lot of research and adoption time. However, as the number of venues was supposed to grow, a new approach to data engineering was needed to provide automation and scalability along with cost optimization.

Tech Tasks Solved with Kubernetes

1. Historical data gathering scripts on a scheduled basis.
2. Data storage in a database that runs inside Kubernetes.
3. AI scripts after successful historical data update.
4. API that interacts with the AI dashboard.
5. An AI dashboard which shows results of the AI scripts.

Business Tasks Solved With Kubernetes

Kubernetes allowed to implement auto-scaling and provided real-time computing resources optimization.

1. Performance and Cost Optimization

It was noticed that on Kubernetes, the AI scripts, which calculated the same logic for the same amount of data, returned results much faster than on AWS EMR, at the same time consuming less computing resources than EMR did. On Kubernetes, the average time required for running the AI module scripts decreased 10 times for the same number of venues, in comparison to the previous EMR production environment.

2. Reliability Improvement

System stability was the key reason that made us switch from AWS EMR to Kubernetes. On EMR, script start failed sometimes for unknown reasons, and the logs didn’t give any helpful information.

3. Scalability Improvement

While on Amazon EMR, the project development was limited by a maximum number of new venues to be added in the future. Kubernetes withdrew the limitation and automated scaling, which is critical for fast-growing projects.

Project Summary

The system on Kubernetes provides results faster, consumes less computing resources, allows the client to reduce costs of the AWS billing and ensures stable and predictable product delivery.

Kubernetes Use Case #2

Data Engineering for an AI Video Surveillance System

Another real-world business use case for Kubernetes MobiDev was involved in, is a smart computing resource auto-scaling for a face blurring feature in a video surveillance system. The system consisted of the following applications: front end, back end, back-end queues, and an AI-based feature for face blurring. Kubernetes was used as the Orchestrator for all these applications.

When new video processing requests appeared, back end auto-scales with the help of Kubernetes API, and automatically added more workers to process the requests.

The Future of Kubernetes

Kubernetes’ distributed architecture and scalability pair well with Machine Learning and Artificial Intelligence. As these technologies continue to mature, 2021 is the year to watch for growth in this space.

Enterprises should keep in mind that tools support business goals, and are not an end to themselves. 2020 was a year in which nearly every business was forced to reckon with unexpected changes. Kubernetes has the capability to accelerate the application development services through solutions built with a cloud-native ecosystem, while at the same time allowing for malleable use of applications and data, so that a company can succeed through the modernization of its platforms and apps.