You have a Shell action which fails. For whatever reason. You need to find the logs of the application. There are essentially 3 logs in oozie The oozie log ( of the server ) The launcher log, which would most likely be interesting for you. You can find it in Hue, the oozie ui ( under the action logs ) or in yarn. This should have some information on what went wrong. ( Look for an oozie launcher in resource manager ) The Application log, not applicable for a shell action but for example for a pig action or other map reduce actions. Also Hue or yarn. ... View more

Just to make sure, when you use a Kafka Consumer on your test topic you see messages coming in? Below is the code of the KafkaWordCount example, the parameters of the Stream look good. ( I would add the port number just to be sure :2181 but he would complain if he couldn't connect to zookeeper. ( although if its a warning you wouldn't see that since you switched off logging ) I also don't trust your function, I know it might be some scala magic and do exactly what it should but it seems needlessly complicated. So my suggestions - Use consumer to test if data comes in - Turn on logging - Add zookeeper port - Replace your map function with the simple .map(_._2) from the Wordcount example - print those and see what happens. val Array(zkQuorum, group, topics, numThreads) = args val sparkConf = new SparkConf().setAppName("KafkaWordCount") val ssc = new StreamingContext(sparkConf, Seconds(2)) ssc.checkpoint("checkpoint") val topicMap = topics.split(",").map((_, numThreads.toInt)).toMap val lines = KafkaUtils.createStream(ssc, zkQuorum, group, topicMap).map(_._2) val words = lines.flatMap(_.split(" ")) val wordCounts = words.map(x => (x, 1L)) .reduceByKeyAndWindow(_ + _, _ - _, Minutes(10), Seconds(2), 2) wordCounts.print() ... View more