Actually, I don't know the exact reasons and had stuck in this problem for a few day with firewalls on all machines disabled at very first. I used to deploy hadoop, spark and so on by extracting source tarballs. Forturnately, edge node seems to be a good idea to acess cluster resources. ... View more

Hi, JasonChen. You have to copy /opt/cloudera/CDH/jars , /etc/hadoop from a node of cluster to your machine runing oryx2. I had tried a few ways to run it outside the cluster, but all failed. The node running oryx2 had to be runed inside cluster. My conclusion is that , CDH maybe requrie the same parcels version and cloudera agent on node to use the cluster resources.     ... View more

you had talk about many issues above, but I find it more related to oryx 1 and MR2. I wonder whether it possible to run oryx2 outside a CDH cluster?   I deployed a hadoop2.6.0-CDH-5.4.4 cluster with zookeeper, kafka , spark on yarn and hdfs. After I tried to run oryx2 on my laptop outside the cluster above(the same CDH version deployed but not running ),  batch layer didn't print out as expected: 2015-08-20 23:45:39,278 INFO  BatchLayer:82 Creating message stream from topic 2015-08-20 23:45:39,531 INFO  AbstractSparkLayer:224 Initial offsets: {[OryxInput,0]=21642186} 2015-08-20 23:45:39,610 INFO  BatchLayer:117 Starting Spark Streaming 2015-08-20 23:45:39,677 INFO  BatchLayer:124 Spark Streaming is running   and it printed out exception at last : Exception in thread "main" java.net.ConnectException: Call From m4040/192.168.88.46 to 0.0.0.0:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused   On batch and speed web page, it showed like this:       I guess my laptop could not communicate with kafka on cluster and this oryx job was rejected by yarn ?!   ... View more