is ranger enabled for Knox? If so make sure you create a ranger knox policy and provide permission for guest user. ... View more

You can do two things - create a Hive user defined function that validates each type. or use PIG to do this function prior to loading the data into Hive. ... View more

@nfakhar, I had recently created historical build for ETL offload to Hive. We added Effective start date and effective end date to each row. I agree with you adding hash function increases performance for full row compare. The newer version of Hive has Hash functions, but we had used datafu for hashing using PIG. ... View more

Is this a new installation? You should start HBASE from Ambari. HBASE requires zookeeper to be available. Paste the error log for HBASE Master. ... View more

Is the datanode service within each slave node appearing started in Ambari?. Also on any one of the slave node that is not working check for errors on /var/log/hadoop/hdfs/<datanodelog> file? Also on slave 1 check namenode log file to see if the datanodes are trying to heartbeat to namenode. Regards Pranay Vyas ... View more

Hi, 1) Number of mappers depends on various factors. primarily number of splits - mapreduce.input.fileinputformat.split.minsize & mapreduce.input.fileinputformat.split.maxsize So a 5GB file configured to have max split size and min split size of 1GB will have 5 mappers. This is just an illustration. See this for Recommended values -> https://community.hortonworks.com/questions/2179/recommended-config-mapreduceinputfileinputformatsp.html 2) Number of containers depends on container size. Read this for calculation of container size http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ 3) Distcp - read this https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_Sys_Admin_Guides/content/ref-7dbacce5-2629-4e31-b143-e20df092f6d5.1.html Regards Pranay Vyas ... View more

Hi Jason, 1) Klist from svchdfs says not ticket cache 2) Klist of keytab shows svchdfs-<clustername>@REALM.COM 3) kinit -kt hdfs.headless.keytab svchdfs-<clustername> We noticed that svchdfs-<clustername> exists at 2 OU's within AD. That could be a cause since kerberos is unable to uniquely identify service account. we are trying to delete the duplicate one. Regards Pranay Vyas ... View more