@ARUN The whole purpose of “balancer” utility is to help balance the blocks across DataNodes in the cluster. So it should do the job, if there is no major issue at the cluster level. It is usually recommend to run the balancer periodically during times when the cluster load is expected to be lower than usual. Also please refer to the following article that explains the importance of balancer and the performance improvement facts: https://community.hortonworks.com/articles/43615/hdfs-balancer-1-100x-performance-improvement.html You can run the HDFS balancer in Maintenance window as well as Without a Maintenance window. Few things you should keep in mind while running the balancer as mentioned in : https://community.hortonworks.com/articles/43849/hdfs-balancer-2-configurations-cli-options.html ... View more

Hi all, Luckily i was able to find the reason and fixed the issue. Just followed the steps in this url http://lecluster.delaurent.com/kill-zombie-dead-regionservers/ Hope it helps ... View more

Luckily i found out the cause. The property i was trying to set is phoenix.schema.isNamespaceMappingEnabled. The property had phoenix. which is a prefix for the phoenix interpreter. so it was not set properly on the interpreter side. so the right property i need to set is phoenix.phoenix.schema.isNamespaceMappingEnabled so the prefix gets correctly parsed. I found out this from the logs JDBCInterpreter.java[open]:142) - key: phoenix, value: schema.isNamespaceMappingEnabled (before the change) JDBCInterpreter.java[open]:142) - key: phoenix, value: phoenix.schema.isNamespaceMappingEnabled (after the change) Hope this helps ... View more

@ARUN You can redirect console output to some file --> grep application ID from that output file --> use yarn command get the job information #Run job [hdfs@prodnode1 ~]$ /usr/hdp/current/hadoop-client/bin/hadoop jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples-2.7.1.2.4.2.0-258.jar pi 10 10 1>/tmp/op 2>/tmp/op & #Grep Application ID [hdfs@prodnode1 ~]$ grep 'Submitted application' /tmp/op |rev|cut -d' ' -f1|rev application_1478509018160_0003 [hdfs@prodnode1 ~]$ #Get status [hdfs@prodnode1 ~]$ yarn application -status application_1478509018160_0003 16/11/11 13:06:07 INFO impl.TimelineClientImpl: Timeline service address: http://prodnode3.openstacklocal:8188/ws/v1/timeline/ 16/11/11 13:06:07 INFO client.RMProxy: Connecting to ResourceManager at prodnode3.openstacklocal/172.26.74.211:8050 Application Report : Application-Id : application_1478509018160_0003 Application-Name : QuasiMonteCarlo Application-Type : MAPREDUCE User : hdfs Queue : default Start-Time : 1478869426329 Finish-Time : 1478869463505 Progress : 100% State : FINISHED Final-State : SUCCEEDED Tracking-URL : http://prodnode3.openstacklocal:19888/jobhistory/job/job_1478509018160_0003 RPC Port : 42357 AM Host : prodnode1.openstacklocal Aggregate Resource Allocation : 129970 MB-seconds, 228 vcore-seconds Log Aggregation Status : SUCCEEDED Diagnostics : [hdfs@prodnode1 ~]$ Hope this information helps! 🙂 ... View more

No. During the start of the MR job you may see message like: mapreduce.MultiHfileOutputFormat: Configuring 20 reduce partitions to match current region count That's exactly the number of reducers that will be created. How many of them will be running in parallel depends on the MR engine configuration. ... View more