Machine Learning: From Monolith to Microservices and Data Flows

Introduction: The Project

A large (really large) locomotive repair facility that is in an ongoing 'digital transformation process.' The goal is to provide a solution to determine the number of locomotives in the facility and their accurate positions. To achieve it, a bunch of IP cameras were deployed to cover all repair stations. Specifics of deployment: no single camera can see the whole locomotive due to the facility's configuration and locomotive size.

In this article, we will not discuss the ML methods that we used in this project. The goal is to show monolith to micro-services transformation and why it's useful.

Stage 1: Proof of Concept (PoC)

Monolith python app that:

• Gets streams from all cameras,
• Determines head or tail positions of locomotives for each camera,
• Combines estimations from all cameras into a facility state estimation (array of all the locomotives positions estimations).

The app is dockerized and deployed to the server with a GPU using NVIDIA Container Toolkit. All cameras are connected to this server via a previously deployed local network.

PoC works and algorithms of locomotive position and facility state estimations need further development. Sometimes the app crashes due to instability work of cameras or other errors.

Stage 2: Microservices

Application is divided into several types of services:

• Video stream capture.
• Image aggregator: receives, processes, caches images, and generates packets for further processing.
• Locomotive position estimator (**LPE**): estimates the position of a locomotive on specific railways.
• Facility state estimator (**FSE**): asynchronously receives locomotive position estimations and estimate facility state 

Services are connected via message/stream processing software. Most popular options here are:

• zeroMQ / imagezmq (less latency)
• rabbitMQ (easy to use, manage and monitor)
• Kafka (need some external storage for images)

Since these services are loosely coupled, teams can be extended with data scientists, developers, and DevOps.

Stage 3: Data Flows

Data from this app is sent to our cloud solution, where it's moved to storage (TSDB) and used for real-time dashboard and reporting. The functionality of this cloud solution is constantly improving, so the real-time flow of data is more than appreciable by developers. The flow of real data with minor flaws is more preferable than the flow of fake data; this lets us include BA and PM into a more motivated feedback loop. 

The first two services - video stream capture and image processor need almost no improvements over time. Once deployed, they stayed on. 

We split data flow into two after the image processor:

• *dev flow : dev LPE + dev FPE*
• *prod flow: prod LPE + prod FPE*

It was enough for that moment but can easily be extended for more flows to test different scenarios (dev LPE + prod FSE, prod LPE + dev FSE) or introduce test/QA loops.

Stage 4: Extensibility

After stabilizing the quality of FSE data flow, a new inquiry from PM came: 'can more information from the video streams be extracted? Like the locomotive model/type, and plate number'? To solve these challenges, two more microservices were created:

• an estimator of the locomotive model,
• 'plate number' recognizer.

They receive images from image processor service queues and send information to FSE.

Conclusion

1. Monolith app is good for simple PoC. It's easier for analysts to create monolith because they used to do it this way. However, as soon as you understand that project is off the PoC stage, stick to microservices and dataflows.
2. Treat image processing as common IIoT data flow :  receive, transform, estimate state, estimate state of the higher-level system.