I am unable to locate /hbase-secure znode , which one should i delete have the same issue , I am just having /hbase znode  ... View more

INFO: Exception in thread "main" java.lang.IllegalArgumentException: Required AM memory Above error is for AM and not for executors, hence you need to set the AM memory as spark.yarn.am.memory=2g ... View more

Hi @loridigia    If cluster/application is not enabled dynamic allocation and if you set --conf spark.executor.instances=1 then it will launch only 1 executor. Apart from executor, you will see AM/driver in the Executor tab Spark UI. ... View more

Hi @loridigia, Based on the current error you provided "org.apache.hadoop.hbase.NotServingRegionException:  table XXX is not online on worker04" maybe some regions are not deployed on any RegionServers yet. please check this result to see is there any inconsistencies on this table: 1. sudo -u hbase hbase hbck -details > /tmp/hbck.txt 2. If you see inconsistencies please grep ERROR from hbck.txt you will see which region has problem. 3. Then you need to check if this region's directory is complete in this result: hdfs dfs -ls -R /hbase 4. Then need to check in hbase shell : scan 'hbase:meta', if this region's info are updated in hbase:meta table. 5. Based on type of the issue we need to use hbck2 jar to fix the inconsistencies. https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2 These are general steps to deal with this kind of problem, there could be more complex issues behind it. We suggest you to file a case with Cloudera support.    Thanks, Will ... View more