I need to save a model in python spark 1.6.0. I know save()/load functions are available in 2.0 but I'm not in a position to upgrade our HDP cluster at this current time and need a hack.
I know Scala 1.6 does support saving of models. Is there some way I could share my model object from python to scala. Im using zeppelin so this might work between paragraphs?
Any help much appreciated
ps - i tried pickling the model object but that didnt work
To understand, if you want to save the model to HDFS, then you can use the model.save functionality. The article below is an interested walk through the entire model process:
You can also save the model into PMML format:
I have checked spark 1.5.0 documentation and model.save(sc,"hdfs path"), <ModelClass>.load(sc,"hdfs path") are supported. Can you give a specific example ?