Maximum number of mappers will be bound by the number of splits calculated at split generation time. These settings impact split calculation: mapreduce.input.fileinputformat.split.minsize mapreduce.input.fileinputformat.split.maxsize Splits are grouped at the Tez layer based on these settings: tez.grouping.min-size tez.grouping.max-size If you want more mappers you can tune all these settings down. Note this will not guarantee lower latency, especially on small clusters. ... View more

Its better not to disturb the properties on the statistics usage like hive.compute.query.using.stats. It impacts the way the statistics are used in your query for performance optimization and execution plans. It has tremendous influence on execution plans, the statistics stored depends on the file format as well. Therefore definitely not a solution to change any property with regards to statistics. The real reason for count not working correctly is the statistics not updated in the hive due to which it returns 0. When a table is created first, the statistics is written with no data rows. Thereafter any data append/change happens hive requires to update this statistics in the metadata. Depending on the circumstances hive might not be updating this real time. Therefore running the ANALYZE command recomputes this statistics to make this work correctly. ... View more

Yes that is recommended approach. ... View more

Hi @Sami Ahmad Normally, master services can be spread across the master nodes to ensure proper resource allocation depending on the cluster. If you have two datanodes/worker nodes that you do not want to run master services on, then, no problem, just allocate the host you want and move on to the next step. In Ambari, you click on the Hosts tab to see what services are installed on what host, but, you may need to go through them host by host. ... View more

found the solution , on web all info about no_proxy settings for Centos is incorrect. what worked for me was removing the ~.wgetrc file and putting the following file in place . issue is that wget is not taking the no_proxy settings from .wgetrc file, but if I define it on the system level it picks it up. /etc/profile.d/proxy.sh export http_proxy="http://dotatofwproxy.tolls.dot.state.fl.us:8080/" export https_proxy="https://dotatofwproxy.tolls.dot.state.fl.us:8080/" export ftp_proxy="ftp://dotatofwproxy.tolls.dot.state.fl.us:8080/" export no_proxy=".tolls.dot.state.fl.us,hadoop1,hadoop2,hadoop3,hadoop4,hadoop5" ... View more

Jay you were right , I removed the agent and reinstalled and it fixed the issue. ... View more

thanks Jay , reinstalling agent fixed this script. I will go ahead and deinstall the cluster using this script and then try to reinstall again. my cluster was running f ine until I deinstalled it and since then I am unable to get it up and running . ... View more