Using SingleStore as a ML Feature Store
SingleStore comes to the Feast
Join the DZone community and get the full member experience.Join For Free
This article quickly shows how SingleStore can be used as an online Machine Learning Feature Store with just a few changes to a code example provided on GitHub.
Feast (Feature Store) is described as follows:
Feast (Feature Store) is an operational data system for managing and serving machine learning features to models in production. Feast is able to serve feature data to models from a low-latency online store (for real-time prediction) or from an offline store (for scale-out batch scoring or model training).
Figure 1 shows the overall flow with Feast.
Figure 1. Feast.
Serving models from a low-latency online store is a perfect use case for SingleStore. Let's see how we can achieve this.
To begin with, we need to create a free Managed Service account on the SingleStore website. At the time of writing, the Managed Service account from SingleStore comes with $500 of Credits. This is more than adequate for the case study described in this article.
Create the Database
In our SingleStore Managed Service account, let's use the SQL Editor to create a new database. Call this feast, as follows:
CREATE DATABASE IF NOT EXISTS feast;
Install and Configure Feast
We’ll start by installing Feast, as follows:
pip install feast
Next, let's clone the GitHub repo:
git clone https://github.com/feast-dev/feast-custom-online-store-demo cd feast-custom-online-store-demo
Now, we'll run the following:
pip install -r requirements.txt
MySQL is the example that is provided for a Custom Online Store. We can adapt this very easily for SingleStore. The example uses MySQL Connector. However, for most Python use cases, SingleStore recommends using PyMySQL. All we need to do is install this, as follows:
pip install PyMySQL
Now, we need to navigate to the following directory:
Modify the Example Application
In the file
mysql.py we need to make a few minor modifications:
- Replace all
Next, in the file
mysql.py we'll modify the following lines:
host=online_store_config.host or "127.0.0.1", user=online_store_config.user or "root", password=online_store_config.password, database=online_store_config.database or "feast",
host=online_store_config.host or "<TO DO>", user=online_store_config.user or "admin", password=online_store_config.password or "<TO DO>", database=online_store_config.database or "feast",
<TO DO> for host and password should be replaced with the values obtained from the SingleStore Managed Service when creating a cluster.
Run the Code
We'll navigate back up one directory level and run:
PYTHONPATH=$PYTHONPATH:/$(pwd) feast -c feature_repo apply
The output should look like this:
Registered entity driver_id Registered feature view driver_hourly_stats Deploying infrastructure for driver_hourly_stats
Next, we'll run the following:
PYTHONPATH=$PYTHONPATH:/$(pwd) feast -c feature_repo materialize-incremental 2021-08-19T22:29:28
The result should look like this:
Materializing 1 feature views to 2021-08-19 23:29:28+01:00 into the feast_custom_online_store.mysql.MySQLOnlineStore online store. driver_hourly_stats from 2020-09-28 19:54:28+01:00 to 2021-08-19 23:29:28+01:00: 0%| | 20%|█████████████ | 1/5 [00: 40%|██████████████████████████ | 2/5 [00: 60%|███████████████████████████████████████ | 3/5 [00: 80%|████████████████████████████████████████████████████ | 4/5 [00:100%|█████████████████████████████████████████████████████ | 5/5 [00:100%|█████████████████████████████████████████████████████ | 5/5 [00:07<00:00, 1.50s/it]
We have reached the end of the example from the GitHub repo. However, if we look at the Quickstart, we can also find some additional steps to follow and explore.
This article showed rapid code modifications to the Feast MySQL Custom Online Store so that Feast could be used with SingleStore instead. However, we did not consider any optimizations. For example, it may be better for performance reasons to consider using a ROWSTORE for the database table. We'll discuss using SingleStore as a Feature Store in more detail in future articles.
Published at DZone with permission of Akmal Chaudhri. See the original article here.
Opinions expressed by DZone contributors are their own.