Oryx 2 with cloudera 5.3

[#1]

Hello Everybody
I am trying to running oryx 2 in Cloudera Express 5.3.0, with a unique node but when i execute the SpeedLayer , this return me the next warning

"WARN YarnClient Cluster Scheduler:71 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory"

I dont know how solve this problem.

[#2 · Accepted solution]

This means that your YARN cluster does not have the amount of resources (CPU, memory) available that the app is asking for. How much is available?

The default is pretty modest though; 2 executors, and 1g RAM and 8 cores per executor. Maybe that's too many cores to ask for? I could turn down the default.

You can change it in your config file.

oryx = {
...
speed = {
  streaming = {
    ...
    # Number of executors to start
    num-executors = 2

    # Cores per executor
    executor-cores = 8

    # Memory per executor
    executor-memory = "1g"

    # Heap size for the Speed driver process
    driver-memory = "512m"
    ...
  }

}

[#3]

Thank you very much the mistake was I was only  checking these attributes in the Batch Layer equal that example configuration.

Now  I have a question , what are the necessary machine resources to run oryx 2 properly?

[#4]

There is no absolute minimum. I would imagine that, for reasoanbly sized problems, you'd want to allocate 1GB of memory to each of these processes -- meaning at least 1 executor for both batch and streaming with 1GB of memory, and at least 1 and probably more cores. The serving layer should probably have 1GB+ of memory too and will use as many cores as the machine has.

This is designed for a cluster of machines, I suppose, but nothing prevents you from running everything on one machine.

[#5]

Hi Nelson,  I am trying to running oryx 2 in Cloudera , do have any suggestions or tutorial? How to begin? 

Thanks 🙂

[#6]

Have a look at this minimal get-started example:
http://oryxproject.github.io/oryx/docs/endusers.html

[#7]

Thanks 🙂