@ARVINDR  There is something you got to investigate. The correct output should look like below. I remember answering such a question something I need to locate the solution     [zk: localhost:2181(CONNECTED) o] ls /hiveserver2 [serverUri=hdp2.test.com:10000;version=1.2.1000.2.6.2.0-205;sequence=0000000061] [zk: localhost:2181(CONNECTED) 1] Can you get the ACL for that znode [zk: localhost:2181(CONNECTED) 0] getAcl /hiveserver2 'world,'anyone : cdrwa [zk: localhost:2181(CONNECTED) 1]   The above is for a non-kerberized cluster . Please revert ... View more

@SwasBigData  I think you missed the outputyou  but you should still be able to locate the log. There is a step during the configure Ranger and DAS Databases that should have give you and output. After running the gen_embedded_ranger_db.sh located in /opt/cloudera/cm/bin on the Cloudera Manager host I will do a trial install tomorrow. ... View more

@npdell  Before starting the upgrade did you by any change validate the upgrade path with Cloudera supportmatrix  this should have been your first reference source. ... View more

@desind  I can see the error  Authentication is not valid  but it seems you didn't use the format super:password->super:DyNYQEQvajljsxlhf5uS4PJ9R28=   instead, your input was as below according to the steps you shared. addauth digest super:password And then  delete  the znode  that should work [zk: xxx.unx.sas.com(CONNECTED) 2] deleteall /kafka-acl/Topic Please do that and revert         ... View more

@ARVINDR  Yes, its possible first questions first Don't attempt this if this is a production cluster  !!! Only when it's your scratch/dev cluster. What is the HDP version? The username mapping is managed by the Isilon admin  I don't know how the authorization works maybe the HDP admin user has been delegated to Isilon where changing it at the cluster level won't synchronize with the OS. After the above response, we can come up with a procedure        ... View more

@desind  By default, Zookeeper runs without the option of becoming a superuser to administrate znodes in the ZK ensemble, for example, to fix ACLs, remove znodes that are not required anymore, or create new ones in specific locations. Zookeeper grants permissions through ACLs through different schemas or authentication methods, such as 'world', 'digest', or 'sasl' if we use Kerberos. We can potentially we locked out if we were to grant everyone just read permissions to a znode, as we would not be able to delete it or modify it anymore.     ... View more

@desind  I tweaked it a little bit it should work in Cloudera Go to Cloudera zookeeper server home   # cd $CDH_HOME/zookeeper-server   Run below command   java -cp "./zookeeper.jar:lib/slf4j-api-1.6.1.jar" org.apache.zookeeper.server.auth.DigestAuthenticationProvider super:password   The output should look like below   SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See <a href="<a href="http://www.slf4j.org/codes.html#StaticLoggerBinder" target="_blank">http://www.slf4j.org/codes.html#StaticLoggerBinder</a>" target="_blank"><a href="http://www.slf4j.org/codes.html#StaticLoggerBinder</a" target="_blank">http://www.slf4j.org/codes.html#StaticLoggerBinder</a</a>> for further details. super:password->super:DyNYQEQvajljsxlhf5uS4PJ9R28=   Copy the super: DyNYQEQvajljsxlhf5uS4PJ9R28=  text and login to Cloudera Manager and goto zookeeper config. Add below to zookeeper-env template config   export SERVER_JVMFLAGS="$SERVER_JVMFLAGS -Dzookeeper.DigestAuthenticationProvider.superDigest=super:DyNYQEQvajljsxlhf5uS4PJ9R28="   Save and Restart Zookeeper and l aunch zookeeper shell on CDH cli   # . /bin/zkCli.sh -server your_server.com   addauth as below Now to removing  the ACL should work Now try to delete an ACL in zookeeper this should work.   addauth digest super:password   Unfortunately, I don't have a CDH sandbox  so you might have to adjust some cmds     ... View more

@ARVINDR  In a usual setup, the hive service should run on a master node and you run hive client on the client nodes ie. edge nodes and data nodes because during deployment Ambari copies the master configuration to all clients host e.g copy hive config to all host where hive client has been installed.   ... View more

@ARVINDR  Why did you have to re-install the service? The issue is with Reason: Error mapping uname 'hive' to uid remember the hive uid is mapped to root. I am not an Isilon expert so I am really handicapped here Please let me know  ... View more

@SwasBigData  I have never deployed on GCP and unfortunately, I am not a partner or running the newer CDP the environments I have and HDP 3.x Goodbye open source everything now behind a paywall catch me up on LinkedIn Geoffrey Shelton  ... View more

@desind    You are not running HDP so are you on MapR or Cloudera? ... View more

@kvinod    I can see the setuid bit (drwxr-s---) was set which alters the standard behavior so that the group of the files created inside said directory, will not be that of the user who created them, but that of the parent directory itself $ ls -lrt /disk1/yarn/nm/usercache total 4 drwxr-s--- 4 mcaf yarn 4096 Feb 24 01:26 mcaf Can you remove the setuid bit as the root user   # chmod -s /disk1/yarn/nm/usercache/mcaf Then rerun   Question1. You don't need to explicitly change file permission when you enable Kerberos, it should work out of the box Question2. You don't need to regenerate a new mcafmerged.keytab just copy it to you other edge nodes it should work as that edge node is also part of the cluster   Please revert         ... View more

@ARVINDR  In addition to @ stevenmatison      switch command which people usually ignore  Add a Hive proxy To prevent network connection or notification problems, you must add a hive user proxy for HiveServer Interactive service to access the Hive Metastore   Steps: In Ambari, select Services > HDFS > Configs > Advanced. In Custom core-site, add the FQDNs of the HiveServer Interactive host or hosts to the value of hadoop.proxyuser.hive.hosts. Save the changes. Can you also set the hive.server2.enable.doAs hive.server2.enable.doAs=true --> Run hive scripts as end user instead of Hive user. hive.server2.enable.doAs=false --> All the jobs will run as hive user.       ... View more

@ARVINDR  The user  hdpuser1 should exist locally, to check whether you have a similar output. Did you run  the below. # useradd hdpuser1 To check run the below # cat /etc/passwd | grep hdpuser1 hdpuser1:x:1000:1000:hdpuser1:/home/hdpuser1:/bin/bash   I am not an expert on Isilon so user mapping is something I need to educate myself o. Do you have some documentation that I could read?           ... View more

@SwasBigData  In the previous step [Setup Database] you must have entered values need at this stage data_analytics_studio_database_host [host where you DAS database is running] data_analytics_studio_database_password [password for the above DAS databases default] Where is your CDP running AWS or Azure? ... View more

@npdell  Curious have you checked the firewalls are off and most important that these first 2 lines are present in your/etc/hosts files on each host. Please uncomment them if they are commented out! 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 # Your host entry below here # 192.168.225.68 [FQDN] [ALIAS] Maybe also  try  to ping and sshing  from one host to another     Please revert             ... View more

@npdell  " No route to host " Signals that an error occurred while attempting to connect a socket to a remote address and port. Typically, the remote host cannot be reached because of an intervening firewall, or if an intermediate router is down. If you are not using static IP's can you check on hosts 192.168.225.165,192.168.225.68 and 192.168.225.171 that  their IP's haven't changed by just running   $ ifconfig   The output should match the IP's in the /etc/hosts table   Please do that and revert   ... View more

@kasa  Can you share a scrambled version of your krb5.conf from both clusters and the auth-to_local of both clusters When copying data from a secure cluster to a secure cluster, the following configuration setting is required in the core-site.xml file:   <property> <name>hadoop.security.auth_to_local</name> <value></value> <description>Maps kerberos principals to local user names</description> </property>   Secure-to-Secure: Kerberos Principal Name Assign the same principle name to applicable NameNodes in the source and destination clusters.   distcp hdfs://hdp-2.0-secure hdfs://hdp-2.0-secure   The SASL RPC client requires that the remote server’s Kerberos principal must match the server principal in its own configuration. Therefore, the same principal name must be assigned to the applicable NameNodes in the source and the destination cluster. For example, if the Kerberos principal name of the NameNode in the source cluster is nn/host1@realm, the Kerberos principal name of the NameNode in destination cluster must be nn/host2@realm, rather than nn2/host2@realm.   Secure-to-Secure: ResourceManager mapping rules When copying between two HDP2 secure clusters, further ResourceManager (RM) configuration is required if the two clusters have different realms. Can you share your hadoop.security.auth_to_local on both clusters, in order for DistCP to succeed, the same RM mapping rule must be used in both clusters. I am assuming the REALMS are TEST.COM and DEV.COM for cluster 1 and 2 respectively   <property> <name>hadoop.security.auth_to_local</name> <value> RULE:[2:$1@$0](rm@.*CLUSTER1.TEST.COM)s/.*/yarn/ RULE:[2:$1@$0](rm@.*CLUSTER2.DEV.COM)s/.*/yarn/ DEFAULT </value> </property>   Can you try that and revert     ... View more

@kvinod  That's great that the initial issue was resolved with the keytab merge, but if I could ask why did you merge all the key tabs to mcafmerged.keytab  it could have been proper to merge only the Hbase and your mcaf keytab , anyway that said your subsequent is a permission issue on the directory   /disk1/yarn/nm/usercache/mcaf.  Can you share the output of  $ ls /disk1/yarn/nm/usercache and   $ ls /disk1/yarn/nm/usercache/mcaf Can you try changing the permission  with the correct group for user mcaf i.e as the root user # chown -R mcaf:{group} /disk1/yarn/nm/usercache/mcaf Then rerun the Terragen command  that should work.   Keep me posted           ... View more

@kasa    Disctcp is used for inter/Intracluster copy but the command you are running is not wrong because you need the source and destination NameNodes. $ hadoop distcp /user/home/test.txt /tmp/ The most common use of DistCp is an inter-cluster copy, where you copy from NameNode1[nn1] to Namenode2[nn2] on 2 different clusters and both clusters should be up and running during the process $ hadoop distcp hdfs://nn1:8020/source hdfs://nn2:8020/destination Where hdfs://nn1:8020/source is the data source, and hdfs://nn2:8020/destination is the destination. This will expand the namespace under /source on NameNode "nn1" into a temporary file, partition its contents among a set of map tasks, and start copying from "nn1" to "nn2". Note that DistCp requires absolute paths.   Personally I think you should use CopyToLocal instead as according to my understanding you are trying to copy a file from hdfs to you local tmp directory   Assuming your directory /user/home/ is in hdfs and you are running the command as HDFS user! This will copy the test.txt from hdfs to local /tmp directory   $ hdfs dfs -copyToLocal /user/home/test.txt /tmp/ And to successfully copy between 2 kerberized cluster you should perform the Kerberos cross-realm trust for distcp it's simple to setup just follow the guide and you will be fine   Please let me know if my assumption is correct   ... View more

@stalsams  If you have the root access just  switch to user sqoop as root user # su - sqoop $ id uid=1020(sqoop) gid=1007(hadoop) groups=1007(hadoop)   Now you should be able to execute all sqoop commands with no need for a password     ... View more

@Arun66  unfortunately with such a vague and incomplete log, we can't help much. Questions? CDH or HWX Shar e the logs? Share the command being executed? Kerberized or not And any indo you deem important   Happy hadooping   ... View more

@chinni  Please elaborate your question else the simplest way to get the latest file in Linux   $ ls -lrt /automation_test/oozie/output/Bigdata_Counts/   But I guess you want to do that programmatically? ... View more

@ARVINDR  This is a permission issue in hdfs, so what you should do is as follow to resolve the issue Assuming you are logged in as root  change recursively the directory owner to [user:group ] hive:hdfs # su - hdfs $ hdfs dfs -chown -R hive:hdfs /tmp/hive Check the ownership now $ hdfs dfs -ls /tmp Desired output The output should match the  below snippet  drwxrwxrwx - hive hdfs 0 2018-12-12 23:43 /tmp/hive Now if you re-run HQL it should succeed. Please let me know ... View more

@kvinod  Your issue can be resolved by merging the keytabs in question. Merge keytab files If you have multiple keytab files that need to be in one place, you can merge the keys with the ktutil command.   Depending on whether you are using MIT or Heimdal Kerberos the process is different but to merge keytab files using MIT Kerberos, use: In the below example I am merging [mcaf.keytab],[hbase.keytab] and [zk.keytab] into mcafmerged.keytab you can merge n number of keytabs but you must ensure the user executing has the correct permissions, it could be a good idea to copy the keytabs and merge them from the users' home directory $ ktutil ktutil: read_kt mcaf.keytab ktutil: read_kt hbase.keytab ktutil: read_kt zk.keytab ktutil: write_kt ktutil: quit To verify the merge Use $ klist -k mcafmerged.keytab Now to access hbase $ sudo kinit -kt mcafmerged.keytab mcaf@Domain.ORG The keytab file is independent of the computer it's created on, its filename, and its location in the file system. Once it's created, you can rename it, move it to another location on the same compute. ... View more

@eistre91    Have you thought of using the HDP Sandbox which is still available for download?  This is the ONLY  environment whose  component versions are a near match to the HDP 3.2.5. It's really unfortunate that everything now is behind a paywall. As a contributor, I have also been hit hard and cannot anymore help on newer versions of the software this is the only occasion I regret the merger. ... View more

@Joe_Jim  By default, the broker binds to localhost. Can you share your server.properties and logs? I am of the opinion you need to update the server.properties listeners=PLAINTEXT://<IP:_TO_CHANGE>:9092 ... View more

@jking  I have a suspicion can you share a tokenized version of your  cluster /etc/hosts ? How many hosts do you have in your cluster and is the DNS working correctly? ... View more