Applying MLflow to an Existing Machine Learning Project

What Is MLflow?

MLflow is an open-source platform to manage the ML lifecycle, including experimentation, reproducibility, deployment, and a central model registry. MLflow currently offers four components:

• MLflow Tracking
• MLflow Projects
• MLflow Models
• MLflow Registry

In this example, we are going to use an existing Keras Tensorflow example and add MLflow support to it. 

Example: Collaborative Filtering for Movie Recommendations” with Keras and Tensorflow 2.0.0

The example we have selected is “Collaborative Filtering for Movie Recommendations” with Keras and Tensorflow 2.0.0. This example is created by Siddhartha Banerjee and the example Source URL is as below: https://keras.io/examples/structured_data/collaborative_filtering_movielens/

The GitHub source for the sample example is here: 

https://github.com/keras-team/keras-io/blob/master/examples/structured_data/collaborative_filtering_movielens.py

Creating Necessary Files for MLflow:

• Download the source code on your local machine. Run the code locally to make sure it works and you understand it

$ python collaborative_filtering_movielens.py

• Create conda.yaml as below:

xxxxxxxxxx

name: tf_keras_recom_example

channels:

 — defaults

 — anaconda

 — conda-forge

dependencies:

 — python=3.6

 — numpy=1.14.2

 — pandas

 — pip:

 — mlflow

 — tensorflow==2.0.0

• Create MLProject as below:

xxxxxxxxxx

name: movielens-recommendations-keras-tf2

conda_env: conda.yaml

entry_points:

    main:

        command: “python collaborative_filtering_movielens.py”

Now you must have 3 files as below:

Editing the Code File for MLflow:

• Add the following python modules

xxxxxxxxxx

import mlflow.tensorflow

mlflow.tensorflow.autolog()

• Comment matplotlib specific code in the project so plotting UI specific thread will not block the code execution.

xxxxxxxxxx

# import matplotlib.pyplot as plt

....

....

# plt.plot(history.history[“loss”])

# plt.plot(history.history[“val_loss”])

# plt.title(“model loss”)

# plt.ylabel(“loss”)

# plt.xlabel(“epoch”)

# plt.legend([“train”, “test”], loc=”upper left”)

# plt.show()

That’s all. Now we will run the code in 2 separate terminal environments. 

Environment 1: Open the MLflow UI

xxxxxxxxxx

$ mlflow ui

Note: Please open your Web Browser with URL http://localhost:5000 to see the MLflow UI. 

Environment 2: You can run the code with MLflow as below to start the project:

xxxxxxxxxx

$ mlflow run .

Once code start running the MLflow UI will show the progress as below:

Source Code for This Project:

https://github.com/Avkash/300seconds/tree/master/tf2-keras-recommendations

Project Walkthrough & Tutorial Video @ YouTube

https://youtu.be/RFtorfGDtJM

That’s all. Please stay safe and stay healthy.