> How can I have only 68 blocks? That depends on how much data your HDFS is carrying. Is the number much less than expected, and not match the output of 'hadoop fs -ls -R /' list of all files? The space report says only about 23 MB used by HDFS, so the number of blocks look OK to me. > Also, when I run hive job, it does not go beyond "Running job: job_1443147339086_0002". Could it be related? This would be unrelated, but to resolve the issue consider raising the values under YARN -> Configuration -> Container Memory (NodeManager) and Container Virtual CPUs (NodeManager) ... View more

(1) - While the CDH codebase does carry the 2.6 initial node-label implementation features, a lot many more changes and enhancements for node-labelling made it upstream only to 2.8, so its a feature still under some development. You can utilise the 2.6 features for certain in CDH 5.4.x, but only via the CapacityScheduler (following the upstream docs), cause the code support does exist in the sources: https://github.com/cloudera/hadoop-common/tree/cdh5.4.7-release/ (2) FairScheduler support is not in upstream yet. We do have node-labelling for FairScheduler on our roadmap for a future release, but I don't have a shareable ETA for it yet. ... View more

The TTL values are stored as Cell-level tags [1]. To retrieve them back from their Cell, fetch the Cell via Get/etc. and then use the available Tags-relevant APIs on the Cell object: http://archive.cloudera.com/cdh5/cdh/5/hbase/apidocs/org/apache/hadoop/hbase/Cell.html#getTagsArray(), and deserialise the array of tags via http://archive.cloudera.com/cdh5/cdh/5/hbase/apidocs/org/apache/hadoop/hbase/CellUtil.html#tagsIterator(byte[],%20int,%20int) [1] - https://github.com/cloudera/hbase/blob/cdh5.4.5-release/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java#L3483-L3486 ... View more

You can edit this via API just as you would with the Pools API, by using JSON edits to the existing property the UI itself writes into. See https://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_props_cdh540_yarn_mr2included_.html#concept_5v9_49n_yk_unique_1__table_kqj_eb1_wk_unique_1, specifically the entry with name "Fair Scheduler Allocations" and description "JSON representation of all the configurations that the Fair Scheduler can take on across all schedules. Typically edited using the Pools configuration UI.". Use http://cloudera.github.io/cm_api/apidocs/v10/path__clusters_-clusterName-_services_-serviceName-_config.html to update this like any other config. Subsequently, call http://cloudera.github.io/cm_api/apidocs/v10/path__clusters_-clusterName-_commands_poolsRefresh.html, to update the pool configs (if you use Dynamic Resource Pools). ... View more

Does your WebSphere app load a custom set of configs to talk to the remote cluster? Are the JHS configs part of the config set, if so? The below properties are all necessary in having the MR2 job register itself with the JHS for post-job persistence - get these property values to precisely match with the working 'hadoop jar' command host's /etc/hadoop/conf/mapred-site.xml: mapreduce.jobhistory.address mapreduce.jobhistory.webapp.address (OR) mapreduce.jobhistory.webapp.https.address yarn.app.mapreduce.am.staging-dir ... View more