@ARVINDR  The user hdpuser1 should exist locally, to check whether you have a similar output. Did you run  the below. # useradd hdpuser1 To check run the below # cat /etc/passwd | grep hdpuser1 hdpuser1:x:1000:1000:hdpuser1:/home/hdpuser1:/bin/bash   I am not an expert on Isilon so user mapping is something I need to educate myself o. Do you have some documentation that I could read?           ... View more

@npdell  Curious have you checked the firewalls are off and most important that these first 2 lines are present in your/etc/hosts files on each host. Please uncomment them if they are commented out! 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # Your host entry below here # 192.168.225.68 [FQDN] [ALIAS] Maybe also  try  to ping and sshing  from one host to another     Please revert             ... View more

@npdell  "No route to host" Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the remote host cannot be reached because of an intervening firewall, or if an intermediate router is down. If you are not using static IP's can you check on hosts 192.168.225.165,192.168.225.68 and 192.168.225.171 that  their IP's haven't changed by just running   $ ifconfig   The output should match the IP's in the /etc/hosts table   Please do that and revert   ... View more

@kasa  Can you share a scrambled version of your krb5.conf from both clusters and the auth-to_local of both clusters When copying data from a secure cluster to a secure cluster, the following configuration setting is required in the core-site.xml file:   <property> <name>hadoop.security.auth_to_local</name> <value></value> <description>Maps kerberos principals to local user names</description> </property>   Secure-to-Secure: Kerberos Principal Name Assign the same principle name to applicable NameNodes in the source and destination clusters.   distcp hdfs://hdp-2.0-secure hdfs://hdp-2.0-secure   The SASL RPC client requires that the remote server’s Kerberos principal must match the server principal in its own configuration. Therefore, the same principal name must be assigned to the applicable NameNodes in the source and the destination cluster. For example, if the Kerberos principal name of the NameNode in the source cluster is nn/host1@realm, the Kerberos principal name of the NameNode in destination cluster must be nn/host2@realm, rather than nn2/host2@realm.   Secure-to-Secure: ResourceManager mapping rules When copying between two HDP2 secure clusters, further ResourceManager (RM) configuration is required if the two clusters have different realms. Can you share your hadoop.security.auth_to_local on both clusters, in order for DistCP to succeed, the same RM mapping rule must be used in both clusters. I am assuming the REALMS are TEST.COM and DEV.COM for cluster 1 and 2 respectively   <property> <name>hadoop.security.auth_to_local</name> <value> RULE:[2:$1@$0](rm@.*CLUSTER1.TEST.COM)s/.*/yarn/ RULE:[2:$1@$0](rm@.*CLUSTER2.DEV.COM)s/.*/yarn/ DEFAULT </value> </property>   Can you try that and revert     ... View more

@kvinod  That's great that the initial issue was resolved with the keytab merge, but if I could ask why did you merge all the key tabs to mcafmerged.keytab  it could have been proper to merge only the Hbase and your mcaf keytab , anyway that said your subsequent is a permission issue on the directory   /disk1/yarn/nm/usercache/mcaf.  Can you share the output of  $ ls /disk1/yarn/nm/usercache and   $ ls /disk1/yarn/nm/usercache/mcaf Can you try changing the permission  with the correct group for user mcaf i.e as the root user # chown -R mcaf:{group} /disk1/yarn/nm/usercache/mcaf Then rerun the Terragen command  that should work.   Keep me posted           ... View more

@kasa    Disctcp is used for inter/Intracluster copy but the command you are running is not wrong because you need the source and destination NameNodes. $ hadoop distcp /user/home/test.txt /tmp/ The most common use of DistCp is an inter-cluster copy, where you copy from NameNode1[nn1] to Namenode2[nn2] on 2 different clusters and both clusters should be up and running during the process $ hadoop distcp hdfs://nn1:8020/source hdfs://nn2:8020/destination Where hdfs://nn1:8020/source is the data source, and hdfs://nn2:8020/destination is the destination. This will expand the namespace under /source on NameNode "nn1" into a temporary file, partition its contents among a set of map tasks, and start copying from "nn1" to "nn2". Note that DistCp requires absolute paths.   Personally I think you should use CopyToLocal instead as according to my understanding you are trying to copy a file from hdfs to you local tmp directory   Assuming your directory /user/home/ is in hdfs and you are running the command as HDFS user! This will copy the test.txt from hdfs to local /tmp directory   $ hdfs dfs -copyToLocal /user/home/test.txt /tmp/ And to successfully copy between 2 kerberized cluster you should perform the Kerberos cross-realm trust for distcp it's simple to setup just follow the guide and you will be fine   Please let me know if my assumption is correct   ... View more

@stalsams  If you have the root access just  switch to user sqoop as root user # su - sqoop $ id uid=1020(sqoop) gid=1007(hadoop) groups=1007(hadoop)   Now you should be able to execute all sqoop commands with no need for a password     ... View more

@Arun66  unfortunately with such a vague and incomplete log, we can't help much. Questions? CDH or HWX Shar e the logs? Share the command being executed? Kerberized or not And any indo you deem important   Happy hadooping   ... View more

@chinni  Please elaborate your question else the simplest way to get the latest file in Linux   $ ls -lrt /automation_test/oozie/output/Bigdata_Counts/   But I guess you want to do that programmatically? ... View more

@ARVINDR  This is a permission issue in hdfs, so what you should do is as follow to resolve the issue Assuming you are logged in as root  change recursively the directory owner to [user:group ] hive:hdfs # su - hdfs $ hdfs dfs -chown -R hive:hdfs /tmp/hive Check the ownership now $ hdfs dfs -ls /tmp Desired output The output should match the  below snippet  drwxrwxrwx - hive hdfs 0 2018-12-12 23:43 /tmp/hive Now if you re-run HQL it should succeed. Please let me know ... View more