question Re: tuning hyperparameters with Oryx 1.0 in Archives of Support Questions (Read Only)

tuning hyperparameters with Oryx 1.0

Jinsu — Wed, 15 Oct 2014 23:03:41 GMT

Hello all,

I want to ask the community some wisdom in tuning the als model parameters with Oryx 1.x.

The computation and serving layer are all in an isolated box for tuning purpose and will use local-computation.

Based on my current understanding of the system, I plan to carry out the task in following manner:

1. prepare a csv file

2. stop oryx, reset/wipe any current als model, start oryx

3. train als model (/ingest)

4. evaluate the model and post the result (a custom code)

5. modify oryx.conf parameter values, go back to step 2

The main difficulty i have is with step 2.

I'm not sure what happens if I were to keep calliing /ingest with the same data and a changed oryx.conf file, so I'm thinking I should stop oryx(both computation and serving) and reset the system to clean slate.

My questions are:

a. Is there an alternative to achieving tuning with Oryx 1.x

b. Do both oryx-computation and oryx-serving need to stop before wiping the contents of $instance.dir?

Re: tuning hyperparameters with Oryx 1.0

Jinsu — Wed, 15 Oct 2014 23:10:04 GMT

http://community.cloudera.com/t5/Data-Science-and-Machine/Question-on-ingest-service/m-p/19718#

suggests that at least oryx-computation server needs to be restarted?

Re: tuning hyperparameters with Oryx 1.0

srowen — Thu, 16 Oct 2014 13:51:39 GMT

Yes, it's very manual. This is not really something it helps you do well. (2.x will do this automatically)

Yes, stop the instances, delete the previous data, then restart. That should be fine.

Re: tuning hyperparameters with Oryx 1.0

Jinsu — Thu, 16 Oct 2014 17:24:31 GMT

Thank you, Sean.