Dear Colleagues,   I've written a SparkStreaming which gets messages from Kafka and put to Hase. The job running fine in local mode but throwa an NullPointerException in yarn-cluster mode.   User class threw exception: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, cloudera01.mlslocal): java.lang.NullPointerException at com.mls.hadoop.messages.kafka.consumer.MlsHadoopJobSparkStreamNewKafkaToHBase.saveToHBase(MlsHadoopJobSparkStreamNewKafkaToHBase.java:163) at com.mls.hadoop.messages.kafka.consumer.MlsHadoopJobSparkStreamNewKafkaToHBase$1$1.call(MlsHadoopJobSparkStreamNewKafkaToHBase.java:140) at com.mls.hadoop.messages.kafka.consumer.MlsHadoopJobSparkStreamNewKafkaToHBase$1$1.call(MlsHadoopJobSparkStreamNewKafkaToHBase.java:113) at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1.apply(JavaRDDLike.scala:332) at org.apache.spark.api.java.JavaRDDLike$$anonfun$foreach$1.apply(JavaRDDLike.scala:332) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at org.apache.spark.util.NextIterator.foreach(NextIterator.scala:21) at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912) at org.apache.spark.rdd.RDD$$anonfun$foreach$1$$anonfun$apply$32.apply(RDD.scala:912) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869) at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1869) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) at org.apache.spark.scheduler.Task.run(Task.scala:89) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Driver stacktrace:   The code snippet is the following:   kafkaStream.foreachRDD(new VoidFunction<JavaPairRDD<String, byte[]>>() { public void call(final JavaPairRDD<String, byte[]> rdd) throws Exception { rdd.foreach(new VoidFunction<Tuple2<String, byte[]>>() { public void call(Tuple2<String, byte[]> avroRecord) throws Exception {       ... View more

Dear Colleagues,   I#m not able to use oozie in Hue, due to oozie_url (liboozie) is not correct (http:///oozie) . Oozie server is running and i've also restarted hue. But Hue don't grab oozie-server ( Oozie Editor/Dashboard The app won't work without a running Oozie server)   If i set the url in "/run/cloudera-scm-agent/process/1122-hue-HUE_SERVER" and restart Hue, then a new file is created without changes.   Do have any solution to set oozie_url or get the stuff working?   Best regards,  Sebastian ... View more

Dear Colleagues   I want to store files up to 150 MB (10% of file set, other 90% of files are <10 MB)  in HBase. Is ist possible or should i take another approach?   Thanks in advance and best regards,  Butkiz     ... View more

Dear Colleages,   I submitted a Spark Streaming job via Oozie and get the following error messages: Warning: Skip remote jar hdfs://quickstart.cloudera:8020/user/oozie/share/lib/lib_20160405235854/oozie/oozie-sharelib-oozie.jar. Halting due to Out Of Memory Error... Halting due to Out Of Memory Error... Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "LeaseRenewer:hdfs@quickstart.cloudera:8020" Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Executor task launch worker-2" Do you have an idea or a solution to prevent these error messages?   Thanks in advance and best regards,  butkiz ... View more

Dear Colleages,   I want to insert a simple String into HBase using spark streaming. The insert should be done inside a rdd.forech function "public void call(Tuple2<String, byte[]> avroRecord)".   Currently I'vo no idea. I'm able to point out the string values but not inserting into HBase.   Thanks in advance and best regards,  Sebastian   ... View more

Dear Colleages,   i want to stream kafka messages into HBase using spark streaming. i set up an java job and run it with "spak-submit ...  yarb client". It seems to be thatkafka messages were consumed but no output is written. No error messages but the following:   INFO storage.BlockManagerInfo: Added input-0-1467044417600 in memory on quickstart.cloudera:46453 (size: 99.0 B, free: 530.3 MB) 16/06/27 09:20:18 INFO scheduler.JobScheduler: Added jobs for time 1467044418000 ms   Do you have any suggestions on this topic?   Many thanks in advance,  Butkiz   ... View more

Dear collegues,   After upgrade to cdh 5.6 there is no hive query result displayed when i ran a hive query in hue. Also, no Protocol of the map reduce Job is displayed. The Job is market as succeded in Job browser.   Do you habe an idea what the reason why?   Tanks in advance, Butkiz ... View more

Dear Colleagues,   In order to run a SSVD in mahout the documents were represented in a tfidf matrix using seq2sparse (the row-index are the doc-ids and the column-index are the dict-id (word-id)).   The input for SSVD is these tfidf-matrix. The output of the SSVD job are the matrices U,S,V (transpose).   How i can interprete this output regarding the original tfidf-matrix? Should i multiplice the original one with U, S or V? What is the conclusion?   Thanks in advance and best regards,  butkiz   ... View more

Hello Colleagues,   if i run a  Mahout DistributedLanczosSolver Job the following error were thrown from the LanczosSolver :   Exception in thread "main" java.lang.IncompatibleClassChangeError: class org.apache.mahout.math.decomposer.lanczos.LanczosSolver$Scale has interface org.apache.mahout.math.function.DoubleFunction as super class I'm using CDH 5.3.2 and the mahout 0.9 packages for CDH 5.3.2.   If i started the job a map reduce job is was finished and the following log message were shown: INFO lanczos.LanczosSolver: 1 passes through the corpus so far... After this, the error message above occurs.   Do you have any ideas or a soloution regarding this issue?   Thanks and regards,  butkiz     ... View more

Hello,   in order to install a cluster using the CM-Wizard with the private key method i get the following error message:    No provider available for Unknown key file   I used a custom user and select the respective .ppk file. If i logged in via ssh from a remote machine with the same credentials, it works.   Do you have any ideas or solution?   Thanks and Regards,  butkiz ... View more

Hello, our operating system is Debian 8.0 and we would like to install Cloudera Manager succesfully using the INstaller. Currently the Installer throws an error because Debian 8.0 is not supported.   Is there an opportunity to run the Installer on Debian 8?   Best Regards, Sebastian     ... View more