Launching H2O Clusters on Different Ports in pysparkling [Code Snippets]
Learn how to launch an H2O machine learning cluster using the pysparkling package with the approrpiate Python code script.
Join the DZone community and get the full member experience.Join For Free
First, install pysparkling in Python 2.7, set up as below:
> pip install -U h2o_pysparkling_2.1
Now we can launch the pysparkling Shell as below:
Launch pysparkling shell:
~/tools/sw2/sparkling-water-2.1.14 $ bin/pysparkling
Here's the Python code script to launch the H2O cluster in pysparkling:
## Importing Libraries from pysparkling import * import h2o ## Setting H2O Conf Object h2oConf = H2OConf(sc) h2oConf ## Setting H2O Conf for different port h2oConf.set_client_port_base(54300) h2oConf.set_node_base_port(54300) ## Gett H2O Conf Object to see the configuration h2oConf ## Launching H2O Cluster hc = H2OContext.getOrCreate(spark, h2oConf) ## Getting H2O Cluster status h2o.cluster_status()
Now, if you verify the sparkling water configuration, you will see that H2O is running on the given IP and port 54300 as configured:
Sparkling Water configuration: backend cluster mode : internal workers : None cloudName : Not set yet, it will be set automatically before starting H2OContext. flatfile : true clientBasePort : 54300 nodeBasePort : 54300 cloudTimeout : 60000 h2oNodeLog : INFO h2oClientLog : WARN nthreads : -1 drddMulFactor : 10
That's it; enjoy!
Published at DZone with permission of Avkash Chauhan, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.