@Lifeng Ai Actually while running "distcp" you should increase the Memory arguments in the command line where you are running the distcp something like following: (When the datasize is too large) # export HADOOP_CLIENT_OPTS="-Xms4096m -Xmx4096m" # hadoop distcp /source /target . Or please define it using 'HADOOP_CLIENT_OPTS' in hadoop-env.sh ... View more

@Shalini Goel Good to know that this issue is resolved. Please mark this thread as "Accepted" so that it will be helpful for others. ... View more

@Elvis Zhang If it is fresh cluster then "Disable Kerberos" and "Enable Kerberos" should be of. Do you see those buttons present ? If those are also not present then it means that the kerberos installation is not yes completed to finalize stage. In that case you can close the current kerberos wizard (if it is already opened and then try enabling kerberos freshly) ... View more

@Shalini Goel It is strange to see the class "org.apache.atlas.hive.hook.HiveHook" in the stackTrace even if you deleted that hook from your hive config. Have your changes made via ambari got reflected properly to the host where you are running hive shell? - Can you please check and share the file " /etc/hive/conf/hive-site.xml" and verify if you have the following emtry in it ? <property> <name>hive.exec.post.hooks</name> <value>org.apache.hadoop.hive.ql.hooks.ATSHook</value> </property> <property> <name>hive.exec.pre.hooks</name> <value>org.apache.hadoop.hive.ql.hooks.ATSHook</value> </property> . ... View more

@Shalini Goel Are you still seeing the same stackTrace which has the following line (even after removing "org.apache.atlas.hive.hook.HiveHook" )? org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:309) at Can you please post the stackTrace again. It might be slightly different. . ... View more

@Elvis Zhang Please check your "krb5.conf" file if it is pointing the correct KDC host & Realm? The default on you can find in "/etc/krb5.conf" on the hosts. Also as you mentioned that you changed the realm name , so after that did you regenerate the keytabs? Ambari UI --> Admin (Tab) --> Kerberos --> "Regenerate Keytabs" Else the existing keytabs might be having old references. Still if it does not work then "Disable and then Enable" Kerberos should take care of this. But i guess regenerating keytabs should be ok. ... View more

@nshetty We can search for "security_type" = Kerberos from teh following API call: http://AMBARI_HOST:8080/api/v1/clusters/CLUSTERNAME?fields=Clusters/security_type ... View more

@Neeraj Singh Apache Hadoop is just a major component of HDP (Hortonworks Data Platform). HDP includes a lots of other components in it. HDP is a suit of components that are tested and certified to work together. Please see the image explaining HDP in the following link: https://hortonworks.com/products/data-center/hdp/ . Where as the HDF (Hortonworks Data Flow) includes the following: HDF includes the following services: NiFi, Storm, Kafka, Zookeeper, Ambari, Ranger . ... View more