Cloudera Community
ramin
user rank
Explorer
My Accepted Solutions
Show All

install hadoop client on unmanaged host

by Explorer in Support Questions
‎03-31-2018 04:25 AM
‎03-31-2018 04:25 AM
Hello,   I am trying to install the Hadoop client components on a host that is not managed by Cloudera Manager. After doing some digging, some people suggested that simply installing the hadoop-client, and adding the site configuration files should do the trick.   But I can't find hadoop-client!   Here is yum repo:     [cloudera-manager] name = Cloudera Manager, Version 5.12.1 baseurl = https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.12.1/ gpgkey = https://archive.cloudera.com/redhat/cdh/RPM-GPG-KEY-cloudera gpgcheck = 1     And the output of yum:     $ sudo yum install hadoop-client Loaded plugins: fastestmirror ... cloudera-manager | 951 B 00:00:00 ... ... cloudera-manager/primary | 4.3 kB 00:00:00 ... ... cloudera-manager 7/7 No package hadoop-client available. Error: Nothing to do Your help is appreciated. ... View more
Labels:

Re: Cloudera - Kerberos GSS initiate failed

by Explorer in Support Questions
‎03-18-2018 06:23 AM
‎03-18-2018 06:23 AM
@Gabre I am glad that your problem is solved. A couple of things: 1) make sure you are using the MIT implementation of Kerberos. 2) It appears that granting extract priviliges need to be done explicitly for each user. (You can't use wildcard.) Please see this, and note the paragraph that begins with "The extract privilege is not included in the wildcard privilege".   I just realized that you are missing the extract privilege in your ACL for the cloudera-scm user. It appears that you need to change the ACL for cloudera admin from admilc to admilce.   I hope that helps you. Cheers. ... View more

Re: inital password for kerberos principals

by Explorer in Support Questions
‎03-17-2018 07:40 AM
‎03-17-2018 07:40 AM
Thank you! That was indeed the problem. You rock! ... View more

Re: inital password for kerberos principals

by Explorer in Support Questions
‎03-16-2018 08:09 PM
‎03-16-2018 08:09 PM
@bgooley, thank you for responding. I did what you suggested, but I am still unable to authenticate against HDFS. See below.   [root@datanode01 process]# kinit -kt 226-hdfs-DATANODE/hdfs.keytab hdfs/datanode01.domain.com@REALM [root@datanode01 process]# klist Ticket cache: KEYRING:persistent:0:krb_ccache_KK2INr6 Default principal: hdfs/datanode01.domain.com@REALM   Valid starting       Expires              Service principal 03/16/2018 22:55:09  03/17/2018 22:55:09  krbtgt/REALM@REALM [root@datanode01 process]# hdfs dfs -ls / 18/03/16 22:55:22 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 18/03/16 22:55:22 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] 18/03/16 22:55:22 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)] ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "datanode01.domain.com/10.0.0.5"; destination host is: "namenode01.domain.com":8020;  Your help is greatly appreciated.   ... View more

inital password for kerberos principals

by Explorer in Support Questions
‎03-16-2018 03:46 PM
‎03-16-2018 03:46 PM
I used Cloudera Manager to enable kerberos. I verified that all the principals for all hosts are created in my Kerberos database and all the keytabs are distributed to all the nodes. But when I try to authenticate using any of the principals, like hdfs, hbase, etc.. I get this:    $ kinit hdfs/hostname Password for hdfs/hostname@REALM: kinit: Password incorrect while getting initial credentials   I don't remember the CM wizard asking me for a password for all the principals. (It just asked for cloudera-scm/admin principal in Kerberos so that it can create new principals)   Does anyone know what the initial password is for the newly-created principals? Or do I have to go and change the passwords for all and redistribute the keytab to all nodes?   Thank you. ... View more
Labels:

Re: Cloudera - Kerberos GSS initiate failed

by Explorer in Support Questions
‎03-16-2018 03:38 PM
‎03-16-2018 03:38 PM
I am having my own kerberos problem, but I thought I'd share this in case it solves your problem. Cloudera stores its own kerberos keytab in the runtime directory. See if you can authenticate against that keytab. If not, then your runtime keytab is not correct and you may have to redistribute the keytab. (requires shutdown of the roles)   Here is the info you need:   1) One a data node, the runtime keytab is located in /run/cloudera-scm-agent/process/XXX-DATANODE/, for example:   # pwd /run/cloudera-scm-agent/process # ls -l */hdfs.keytab -rw------- 1 hdfs hdfs 1570 Mar 14 23:25 166-hdfs-DATANODE/hdfs.keytab -rw------- 1 hdfs hdfs 1570 Mar 15 20:28 197-hdfs-DATANODE/hdfs.keytab -rw------- 1 hdfs hdfs 1570 Mar 15 21:33 203-hdfs-DATANODE/hdfs.keytab -rw------- 1 hdfs hdfs 1570 Mar 16 18:07 207-hdfs-DATANODE/hdfs.keytab   2) Use kinit to authenticate against the keytab.   # kinit -t hdfs.keytab user/host@realm   If you can successfully authenticate against that keytab, then your keytab is good. I hope this helps. If not, you'll have to redistribute the keytabs.   Good luck.     ... View more

Re: Cloudera manager fails to start trying to fetc...

by Explorer in Support Questions
‎03-06-2018 12:15 PM
‎03-06-2018 12:15 PM
The issue is resolved. The exceptions were misleading; the real issue was this:   Exception in thread "MainThread" java.lang.IllegalArgumentException: Invalid value set for db.setupType, the valid values are EMBEDDED or EXTERNAL I changed the default value for com.cloudera.cmf.db.sertupType in /etc/cloudera-scm-server/db.properties to EXTERNAL. It was set to INIT. ... View more

Cloudera manager fails to start trying to fetch pa...

by Explorer in Support Questions
‎03-06-2018 11:39 AM
‎03-06-2018 11:39 AM
I am doing a manual install, I was able to add yum repo to install Cloudera Manager (CM), but when I try to start it, it show that it started successfully, but the logs show that it fails to fetch the Parcels from external repo. (logs below)   The nodes are on a VPC (in AWS) and yum is configured to use a proxy, but evidently the proxy setting in /etc/yum.conf is ignored. (which is understandable because that is probably used by yum command line and not the Java process.)   So here is my question, is there a way to instruct the CM Server to use a proxy to fetch the parcels? I tried setting HTTP_PROXY and HTTPS_PROXY env variable, but no luck. I am trying to avoid setting up a local repository.   Thank you and here is exception from logs: (this happens for all the parcels it tries to fetch, such as CDH, Kafka, Spark, etc... I only included the log for Sqoop)   2018-03-06 19:04:47,221 ERROR ParcelUpdateService:com.cloudera.parcel.components.ParcelDownloaderImpl: Error while attempting to retrieve repository info for repo https://archive.cloudera.com/sqoop-connectors/parcels/latest/ java.io.IOException: Closed at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.doConnect(NettyAsyncHttpProvider.java:873) at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.execute(NettyAsyncHttpProvider.java:858) at com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:512) at com.ning.http.client.AsyncHttpClient$BoundRequestBuilder.execute(AsyncHttpClient.java:234) at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfoFuture(ParcelDownloaderImpl.java:579) at com.cloudera.parcel.components.ParcelDownloaderImpl.getRepositoryInfo(ParcelDownloaderImpl.java:532) at com.cloudera.parcel.components.ParcelDownloaderImpl.syncRemoteRepos(ParcelDownloaderImpl.java:346) at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:453) at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:448) at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)   ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎05-05-2018 12:32 PM
Community Statistics
Member Since ‎03-06-2018 11:08 AM
Last Visited ‎05-05-2018 12:32 PM
Posts 8