Belarusian AI Service Beats Google and Microsoft AI in Car Recognition

Technology develops at an incredibly fast pace, and the one, who is able to get ahead of their time and to set the direction, wins. Object recognition seemed utopian a few years ago and those, who started working on this matter 2-3 years ago, now have illustrative and remarkable results. The big players in the object recognition industry, that first come to mind, are Google and Microsoft. These are the companies that the user will start looking for in the first place in the object recognition field.

Choosing a direction is the main issue. What to detect? In a way, that will allow it to be sold and purchased, to be used as a service. Our hypothesis is that the number of cars increases with the constantly growing population and metropolitan areas on Earth. The Earth has a limited amount of space, and every other driver, already, faces the challenge of finding a parking spot in the city. Business fights over space and instead of increasing parking areas, shrinks as much as possible and attempts to look for new solutions.

Thus, we needed to solve the application challenge – to detect cars in the pictures, highlight them and report their coordinates. First and foremost, the idea came out of a personal need – to quickly and effectively find and see available parking space in the yard before the driver arrives there. Later, the idea was transformed into a business goal – to offer additional service for clients of any company that owns a parking lot, as well as clients of video surveillance services.

The goal was to detect the cars in the images from video surveillance and to report the collected data in a schematic way on a map via a mobile app or a computer.

First of all, we tested Google AI, since we trust many Google’s products. And it seemed that for such a big and influential beast, car recognition would be extremely easy to handle. However, disappointment followed. Google AI car recognition has low accuracy. That is why it functions more like PR for other Google products, than a quality system of computer vision.

For the fist test, we used well-lit pictures from optimal height and scale for cars. The trial run was unsuccessful. Out of more than 220 cars, Google AI highlighted only 10.

Test 1. Google AI

Having determined that even the almighty Google fails at such a basic level, we tried Microsoft Computer Vision service to compare results. Here’s the result.

Test 1. Microsoft Computer Vision

Only 7 (!) cars. And this is quite disappointing.

Realizing that AI field has only started to develop, and for the sport of it, we have started collecting a car dataset and train our own neural network. We took the YOLOv3 Darknet framework as a basis. Trials have shown that it produces the least amount of issues during the process. It quickly processes images and automatically applies augmentation if there is an insufficient number of images. After a period of intensive training, we have a trained neural network based on more than 25 thousand images. Currently, we are adding extra 2500 images for training.

Here is how the Belarusian AI system SpotVision Car Detection handled the challenge.

Test 1. SpotVision Car Detection

As you can see, the majority of cars, not only in the parking lot, have been identified. So, where the human eyes can properly see the car, the SpotVision system does a great job.

Let’s make it more difficult and take the night vision. In terms of relevance, it’s more challenging to find a parking spot at night due to low visibility and a big number of parked cars. That is why the night detection is more relevant.

Here are Google’s results.

Test 2. Google AI

And here is Microsoft Computer Vision.

Test 2. Microsoft Computer Vision

That is to say – completely nothing and no cars in the images.

Let’s compare what SpotVision has detected.

Test 2. SpotVision Car Detection

Let’s move forward. Video surveillance cameras are often installed on the top floors of multistoried buildings. This provides the biggest coverage of the parking lot with 1 camera. This allows to reduce expenses of the video surveillance services for equipment and installment. Since the most frequent and popular case is car detection in residential complexes’ and business centers, we have taken upon ourselves to analyze it in three systems. 

Test 3. Google AI

Test 3. Microsoft Computer Vision

Test 3. SpotVision Car Detection

Now, to set the record straight: how well do the systems deal with the challenge if there is a whole car and bits of other cars in the shot?

Test 4. Google AI

Test 4. Microsoft Computer Vision

Test 4. SpotVision Car Detection

Google AI has been able to detect 4 objects, three of them are wheels, not whole cars. Microsoft Computer Vision has been able to detect 2 cars. SpotVision detected 4 whole cars and only one wheel separately. The Belarusian Car Detection System has been able to complete the challenge best, but with a few stipulations. 

It would be quite interesting to know what will the systems be able to highlight in the images, where no obvious car features, that the systems so often rely on in car detection, are presented. These are the features like the standard outline of the car and the wheel shape.

Test 5. Google AI

Test 5. Microsoft Computer Vision

Test 5. SpotVision Car Detection

Surprisingly, all three services have been able to detect the car accurately, only Google AI additionally detected the wheel or colored fume swirls, highlighting them with a rectangle.

Another natural obstacle in car detection is fog and rain. They reduce image’s clarity and might make some objects invisible for computer vision.

Test 6. Google AI

Test 6. Microsoft Computer Vision

Test 6. SpotVision Car Detection

Google AI and SpotVision Car Detection have shown the best results, even though haven’t detected two cars, passing by far away. Microsoft Computer Vision hasn’t detected one car and highlighted only one out of 4 obvious objects.

So, two control tests with a maximum number of cars in the image. There are 46 cars in the Test 7.

Test 7. Google AI

Test 7.Microsoft Computer Vision

Test 7. SpotVision Car Detection

Google AI have detected 11 cars out of 39, Microsoft Computer Vision – 30 cars and highlighted a sign by mistake, and SpotVision detected 46 cars.

Let’s proceed with examples that have a lot of cars. Let’s change the viewpoint.

Test 8. Google AI

Test 8. Microsoft Computer Vision

Test 8. SpotVision Car Detection

            Adding images from real places, taking into consideration seasonal changes. The example provided below depicts winter and snow. As you can see, bare patches and spaces where cars were, but left, can be mistakenly detected as cars. Additionally, cars hidden under snow become invisible to the computer vision.

Test 9. Google AI

Test 9. Microsoft Computer Vision

Test 9.SpotVision Car Detection 

            The same view, night-time. 

Test 10. Google AI

Test 10. Microsoft Computer Vision

Test 10. SpotVision Car Detection

It appears that Google AI and Microsoft Computer Vision operate with a bigger range of tasks, but fail in car detection. They failed most of the cases either fully or partly, in some they made mistakes. That is why they are not suitable for business, that would want to expand their opportunities via integration with AI-services.  Meanwhile, the Belarusian service SpotVision Car Detection has been able to handle the challenges at 98% success rate, and it is ready for practical use in real-time.

If you would like to check each services’ performance, here are the links:

Google AI: https://cloud.google.com/vision

Microsoft Computer Vision: https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/#features

Spotvision Car Detection: https://dashboard.spotvision.co/#/image-load