That generally means it's still waiting for YARN to allocate an executor, and that in turn usually means you don't have enough resources free in YARN to satisfy the request. Check your number and size of executors vs available resources and max size of any one container that your YARN config allows. ... View more

What do you mean by part API? ... View more

Oryx uses Spark Streaming, and Spark runs its executors on YARN. So YARN manages the resources used by the batch and speed layer. You can also use YARN to run the serving layer binaries via the oryx-run.sh script. ... View more

There shouldn't be any other dependencies. If the error is like what you showed before, it's just firewall/port config problems. ... View more

Yes, it uses Spark streaming for the batch and speed layers. Really big models are just 'passed' to the topic as an HDFS location. The max is configurable but is about 16MB. This tends to only matter for decision forests or ALS models with large numbers of users and items. The data in Kafka topics is replicated according to the topic config. Yes it can potentially be replicated across the machines that server as brokers. ... View more

Right, I forgot to mention that part: you need the cluster's binaries too, like ZK, HDFS, YARN, Spark, etc. It is using the cluster's distribution.  As you can see, it's definitely intended to be run on a cluster edge node, so I'd strongly suggest running it that way. ... View more

You can run the binaries on any machine that can see the Hadoop configuration on the classpath, and which can access all of the services it needs to in the cluster. There are a number of services to talk to: HDFS, YARN, Kafka, Spark and the app's executors. So in general you'd have to have a lot of ports open, and at that point your machine is effectively a gateway node in the cluster. Certainly it's meant to be run within the cluster.   The serving layer only needs access to Kafka, and that's by design, so it might more easily run outside the cluster. ... View more