Since Oozie lacks knowledge of where your HBase configs lie, you will need to pass the client hbase-site.xml file (placed somewhere on HDFS, by copying from /etc/hbase/conf/hbase-site.xml on any HBase gateway node) via the <job-xml>…</job-xml> option. Alternatively, try the below command instead (will not be sufficient for secured clusters, which need further properties), replacing zk-host1,zk-host2,zk-host3 with your actual 3 hosts appropriately: sqoop import -Dhbase.zookeeper.quorum=zk-host1,zk-host2,zk-host3 --connect jdbc:oracle:thin:@XXX:port/XXX --username XXX --password XXX --table XXX -m 1 --incremental lastmodified --last-value '2014-06-23' --check-column XXX --append --hbase-table XXX --column-family info --hbase-row-key XXX --hbase-bulkload ... View more

Please post your cluster's memory configuration, such as the resource MB offered by the NodeManagers, and individual MapReduce settings of AM, Map and Reduce task memories. It appears that the cluster's unable to schedule more than 1 or 2 containers at a time, causing the job to eternally hang cause Oozie runs 2x AMs grabbing 2x containers already. ... View more

Yes, the reason of the 200k default is to warn you that you may be facing a small files issue in your cluster, or that you may be close to requiring to expand further horizontally. Having more number of blocks raises the heap requirement at the DataNodes. The threshold warning exists to also notify you about this (that you may need to soon raise the DN heap size to allow it to continue serving blocks at the same performance). With CM5 we have revised the number to 600k, given memory optimisation improvements for DNs in CDH4.6+ and CDH5.0+. You can feel free to raise the threshold via the CM -> HDFS -> Configuration -> Monitoring section fields, but do look into if your users have begun creating too many tiny files as it may hamper their job performance with overheads of too many blocks (and thereby, too many mappers). ... View more

If CM does not present a UI field for an advanced tuning property, you can rely on the Configuration Snippet (Safety Valve) fields to set them in manually. More on this at: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/4.8.1/Cloudera-Manager-Managing-Clusters/cmmc_safety_valve.html ... View more

You receive the error because the 'hbase' user does not have a login shell assigned to it. You can set a shell for the 'hbase' user on the machine, to allow direct 'su' based login to that user, by following http://www.cyberciti.biz/faq/howto-set-bash-as-your-default-shell/ However, if your goal is to simply use the 'hbase' user for running superuser level commands, we instead recommend using 'sudo' style commands. For example: ~> sudo -u hbase hbase hbck ~> sudo -u hbase hbase shell You can also invoke a shell as the 'hbase' user in certain cases, via: ~> sudo -u hbase /bin/bash ... View more

Your java program needs to include cluster client configs on its classpath for the Configuration class to be able to read and discover the actual MR cluster automatically. Typically you can achieve this by adding the directory /etc/hadoop/conf to your classpath, if you are not launching your custom application using the 'hadoop jar' command (which auto-sets the desirable classpath). ... View more