Kubeflow in IBM Kubernetes Services (IKS)
Kubeflow in IBM Kubernetes Services (IKS)
Kubeflow makes life simpler by allowing deployment of Machine Learning workflows on Kubernetes. Learn more in this tutorial!
Join the DZone community and get the full member experience.Join For Free
Kubeflow makes life simpler by allowing deployment of Machine Learning workflows on Kubernetes. This brings in the flexibility of scale and portability.
In this tutorial you will learn:
- Provisioning of a Kubernetes cluster in IBM Cloud and installing the required tools
- Installing Kubeflow
- Launching the Kubeflow dashboard
To run this tutorial, you need:
- An Ubuntu 18 machine with a minimum 8 cores, 16 GB RAM, and 250 GB storage
- Root privilege on the ubuntu machine
- IBM Cloud account
IBM Cloud UI
Log in to IBM Cloud ( https://cloud.ibm.com ). Go to Catalog, under services you can find Kubernetes service.
In this demo, I have used the Kubernetes 1.16.14 (Stable) version in a classic infrastructure in a single zone region. Once you choose the version, select Geography ( Asia Pacific), Availability (Single Zone), Worker Zone (Chennai 01)
The worker node size used in this setup was 4 vCPUs and 16 GB Memory with 1 worker node per zone.
The master service endpoint is set to Public endpoint only.
Choose the cluster name wisely, you can leave it default but recommend to use something of your choice.
Create the cluster
Till the cluster setup is up, log in to the Ubuntu Machine
Download the necessary tools for IBM Cloud.
Login in to your IBM Cloud Account, the region, and resource group might change based on your selection.
Set the Kubernetes context to your cluster for this terminal session.
Verify that you can connect to your cluster.
Note: In case you forget your cluster setting, you can retrieve this information by visiting https://cloud.ibm.com/kubernetes/clusters and clicking Access under your cluster.
To verify your setting, run the following code.
The following should be your output.
When using the classic worker nodes provider of IBM Cloud Kubernetes cluster, by default, it uses IBM Cloud File Storage based on NFS as the default storage class. File Storage is designed to run RWX (read-write multiple nodes) workloads with proper security built around it. Therefore, File Storage does not allow group securityContext which is needed for DEX and Kubeflow Jupyter Server.
IBM Cloud Block Storage provides a fast way to store data and satisfy many of the Kubeflow persistent volume requirements such as fsGroup out of the box and optimized RWO (read-write single node) which is used on all Kubeflow’s persistent volume claim.
Check if there is a default storage class setup to Block Storage.
If not, we need to set up IBM Cloud storage.
Follow the instructions to install the Helm version 3 client on your local machine.
Add the IBM Cloud Helm chart repository to the cluster where you want to use the IBM Cloud Block Storage plug-in.
The following code shows the sample output.
Set the block storage as default, ensure that there is only one default storage
Installing Kubeflow v.1.1.0
You can create your directory, I have created a kubeflow directory under root.
When the installation is complete which might take a few mins, please ensure that all pods are running. Only then proceed to the next steps. If not you might have to recreate this exercise.
Verify if all pods are running:
The sample output below:
Launching the Kubeflow Dashboard
Kubeflow Dashboard can be accessed through the istio-ingressgateway service. To see your setting of istio-ingressgateway service, execute the following code.
You might get the following output:
If your output has a NodePort TYPE (not LoadBalancer) and EXTERNAL-IP is blank like above, run the following command to change the type from NodePort to LoadBalancer.
Run the following command again to get the EXTERNAL-IP.
Now you can access the Kubeflow dashboard in a web browser: http://<EXTERNAL-IP>:<PORT>
That's all folks! In this tutorial, I have explained to you how to set up Kubeflow on IBM Kubernetes Service (IKS). I hope you liked it.
Published at DZone with permission of Deepak Rai . See the original article here.
Opinions expressed by DZone contributors are their own.