Over a million developers have joined DZone.

A Practical Way to Think About Prediction Accuracy

DZone's Guide to

A Practical Way to Think About Prediction Accuracy

It's not uncommon for companies to work on gut feelings. But how accurate the gut feeling is is typically not measured.

· AI Zone ·
Free Resource

The most visionary programmers today dream of what a robot could do, just like their counterparts in 1976 dreamed of what personal computers could do. Read more on MistyRobotics.com and enter to win your own Misty. 

Image title

One of the common questions that gets asked by management when trying to deploy is, "What is the accuracy?" That is the trap companies tend to get into for wanting the best accuracy to go live.

When talking about accuracy, it's important to compare the accuracy that your model provides in comparison to what you do now without the model.

Image title

It's not uncommon for companies to work on gut feelings. But how accurate the gut feeling is is typically not measured.

Many companies currently operate with gut feelings, which could be random, as they typically do not measure the accuracy of their gut feeling.

To compare apple to apples, train a model to answer the predictive question, and get the answer from the model. Get the same answer based on a gut feeling. Now compare the accuracy of the two with the actual outcome.

Identify what the accuracy is based on the current gut feeling estimates, and if the predictive model is performing much better than the gut feeling, you have a winner and can move forward. Continue to tweak the predictive model over time to make it better.

For example, say you are trying topredict who will churn. Maybe your starting accuracy is 65%, but that could be an order of magnitude better than the current guesses you are making. In that case, you can take this model live. Figure out how to distribute this information to the right people and incorporate workflows inside the application your end users use to take action based on this insight. Listen to the data and your users, and continue to improve the model, your distribution strategy, and the action you take.

Image title

Don't wait for all the stars to align. Get started with a predictive problem with a reasonable accuracy, and embed that insight into your application to provide end users a key action they can take within the application. You will learn a lot from your users. You have a baseline and can continue to improve the model and how it gets used every month to pleasantly surprise your users.

Once you’ve started demonstrating the ROI of answering that problem, over time, you can add more data to improve the model and incorporate new insights into other parts of the business workflows. Success with these initiatives will provide your business with a competitively differentiated application and will drive more revenue.

Robot Development Platforms: What the heck is ROS and are there any frameworks to make coding a robot easier? Read more on MistyRobotics.com

data science ,machine learning ,artificial intelligence ,prediction accuracy ,gut feeling

Opinions expressed by DZone contributors are their own.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}