Cloudera Community
jsensharma
user rank
user rank
Master Mentor

Re: how to create new keytab if my previous keytab...

by Master Mentor Master Mentor in Support Questions
‎02-04-2020 02:27 AM
‎02-04-2020 02:27 AM
@OmThakare  Are you managing keytabs/kerberos using Ambari? If yes, then you can regenerate keytabs from Ambari UI. Ambari allows to either regenerate all keytabs or only for specific components on specific hosts.   Regenerating keytabs for all hosts in the cluster is a disruptive operation, and requires all components to be restarted. Optionally, keytabs can be regenerated only for missing hosts and components, and this operation requires selectively restarting those affected hosts and services.   https://docs.cloudera.com/HDPDocuments/Ambari-2.7.4.0/managing-and-monitoring-ambari/content/amb_regenerate_kerberos_keytabs_from_ambari_web.html . If you are not not using Ambari to manage kerberos then  Following link describes How to create service principals and keytab files https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/security-reference/content/kerberos_nonambari_creating_service_principals_and_keytab_files_for_hdp.html   ... View more

Re: How to change user password in CDSW?

by Master Mentor Master Mentor in Support Questions
‎02-03-2020 06:10 PM
‎02-03-2020 06:10 PM
@johnwook  Following links might be useful:   1.  https://docs.cloudera.com/documentation/data-science-workbench/1-0-x/topics/cdsw_external_authentication.html#debug_login_url 2. https://community.cloudera.com/t5/Support-Questions/CDSW-1-6-1-default-Admin-Site-administrator-Username-and/td-p/281599 ... View more

Re: Secure Webhdfs in Hadoop Hortonworks Cluster

by Master Mentor Master Mentor in Support Questions
‎02-03-2020 01:03 PM
1 Kudo
‎02-03-2020 01:03 PM
1 Kudo
@asmarz  Good to know that your original issue is resolved.  However for any subsequent slightly different issue it is always better to open a new Community Thread that way the readers of this thread can easily find out One Error/Issue with one Solution.    Multiple issues in a single thread can cause readers to get confused. . If your question is answered then, Please make sure to mark the answer as the accepted solution. If you find a reply useful, say thanks by clicking on the thumbs up button.   ... View more

Re: AMBARI REST API + how to verify if specific se...

by Master Mentor Master Mentor in Support Questions
‎02-03-2020 12:57 PM
2 Kudos
‎02-03-2020 12:57 PM
2 Kudos
@mike_bronson7 If you just want to find out specific services like HDFS / HIVE /SPARK2 to know whether they require restart or not then you can do this using filtering option service_name.in(xxx,yyy,zzz)   # curl -u admin:admin -H "X-Requested-By: ambari" -X GET "ttp://kerlatest1.example.com:8080/api/v1/clusters/KerLatest/host_components?HostRoles/stale_configs=true&HostRoles/service_name.in(HIVE,HDFS,SPARK2)"     Various Options to get Stale Config Component/Service details. Similarly there are few other variants of the API call to find out and list the Services/Individual components which require restart.   # curl -u admin:admin -H "X-Requested-By: ambari" -X GET "ttp://kerlatest1.example.com:8080/api/v1/clusters/KerLatest/host_components?HostRoles/stale_configs=true&fields=HostRoles/service_name,HostRoles/host_name" # curl -u admin:admin -H "X-Requested-By: ambari" -X GET "ttp://kerlatest1.example.com:8080/api/v1/clusters/KerLatest/host_components?HostRoles/stale_configs=true" # curl -u admin:admin -H "X-Requested-By: ambari" -X GET "ttp://kerlatest1.example.com:8080/api/v1/clusters/KerLatest/host_components?HostRoles/stale_configs=true&fields=HostRoles/service_name"   How to restart All Stale Configurations using API call.   # curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d '{"RequestInfo":{"command":"RESTART","context":"Restart all required services","operation_level":"host_component"},"Requests/resource_filters":[{"hosts_predicate":"HostRoles/stale_configs=true"}]}' "ttp://kerlatest1.example.com:8080/api/v1/clusters/KerLatest/requests"   . Please change the Ambari Hostname/port and Cluster name in the above API calls. I have changed "http://" to "ttp://"  Please correct that protocol prefix before using the above mentioned API calls. ... View more

Re: Secure Webhdfs in Hadoop Hortonworks Cluster

by Master Mentor Master Mentor in Support Questions
‎01-31-2020 03:25 PM
‎01-31-2020 03:25 PM
@asmarz  In order to clarify the port access, From Ambari host please check if the NameNode port and address is accessible? # nc -v $ACTIVE_NAMENODE_FQDN 8020 (OR) # telnet $ACTIVE_NAMENODE_FQDN 8020 The error which you posted usually indicates that before running the mentioned HDFS command you did not get a Valid kerberos ticket using "kinit" command. 20/01/31 16:31:19 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] . Most Possible Cause of above WARNING: If the port is accessible then please check if you are able to run the same hdfs command after getting a valid kerberos ticket. # klist -kte /etc/security/ambariservername.keytab # kinit -kt /etc/security/ambariservername.keytab ambariservername@REALM.COM # klist # export HADOOP_ROOT_LOGGER=DEBUG,console # hdfs dfs -ls / . And then try the same command using the "hdfs" headless keytab # kdestroy # klist -kte /etc/security/keytabs/hdfs.headless.keytab # kinit -kt /etc/security/keytabs/hdfs.headless.keytab hdfs-ker1latest@EXAMPLE.COM # klist # export HADOOP_ROOT_LOGGER=DEBUG,console # hdfs dfs -ls / *NOTE:* the "hdfs-ker1latest@EXAMPLE.COM" principal name may be different in your case so replace it with your own hdfs keytab principle Please share the output of the above commands. Also verify if all your cluster nodes has correct FQDN.   . . ... View more

Re: Ambari rest api automation

by Master Mentor Master Mentor in Support Questions
‎01-31-2020 02:29 AM
‎01-31-2020 02:29 AM
@Ioann  Your API call does not look correct... the JSON data which you are posting is not right.   Please try this kind of request.    # curl -k -i -u admin:password -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop NodeManager","operation_level":{"level":"HOST_COMPONENT","cluster_name":"DEV","host_name":"slave01","service_name":"YARN"}},"Body":{"HostRoles":{"state":"INSTALLED"}}}' https://ambari_server:8443/api/v1/clusters/DEV/hosts/slave01/host_components/NODEMANAGER     . If it still fails in one of the env then please share the exact details about those envs:   1. Exact ambari server version from both the Envs:   # ambari-server --version # ambari-server --hash   2. The Before making the API call on the failing cluster please make sure to put the ambari-server.log in "tail" mode so that we can see what kind of logging is appearing in amabri logs when you get  following kind of INFO message  Ignoring ServiceComponentHost as operation is not allowed, 3. Are you running this API call on both the envs as Ambari Admin user?         ... View more

Re: ClassNotFoundException Spark2-submit

by Master Mentor Master Mentor in Support Questions
‎01-31-2020 01:17 AM
‎01-31-2020 01:17 AM
@ChineduLB  Are you able to get this working? ... View more

Re: Error Hive in Hue - MetaException(message:Got ...

by Master Mentor Master Mentor in Support Questions
‎01-30-2020 05:49 PM
‎01-30-2020 05:49 PM
@Strabelli  As you see the message in the browser as following which is very generic error and does not tell the actual cause of the failure. MetaException(message:Got exception: org.apache.hadoop.hive.metastore.api.MetaException Exception thrown when executing query)   - So did you check the Hue Log to see if it is showing a detailed StackTrace of the error?   Just tail the Hue logs and then try running the same query again to see if we get a detailed error message. # ls -lart /var/log/hue/ # tail -f /var/log/hue/* Also can you check the HIve Service / Metastore log of the same time stamp to verify if the Metastore is running fine without any error?   ... View more

Re: postgresql + ambari server failed to open port...

by Master Mentor Master Mentor in Support Questions
‎01-30-2020 04:14 PM
1 Kudo
‎01-30-2020 04:14 PM
1 Kudo
@mike_bronson7  Based on your details i do not think that your Postgres runs on 8080 port (that may be your Ambari Server port) So if you want to use "psql" utility to connect to Postgres Database then you should be using Postgres Port (default 5432).  Example: # psql --host=`hostname -f` --port=5432 --username=ambari --dbname=ambari Password for user ambari: bigdata ambari=> select * from host_version; # psql --host=localhost --port=5432 --username=ambari --dbname=ambari Password for user ambari: bigdata ambari=> select * from host_version; . In order to know which port your Postgres is listening you can try to look at the followig places. # grep port /var/lib/pgsql/data/postgresql.conf # ps -ef | grep pgsql | grep ^postgres # netstat -tnlpa | grep 5432 # netstat -tnlpa | grep $POSTGRES_PID . Similarly you can confirm what is the Port in which your Ambari Server is listening for HTTP/HTTPS requests. # grep -e 'client.api.ssl.port\|port' /etc/ambari-server/conf/ambari.properties | grep api Example: # netstat -tnlpa | grep `cat /var/run/ambari-server/ambari-server.pid` tcp6 0 0 :::8440 :::* LISTEN 24715/java tcp6 0 0 :::8441 :::* LISTEN 24715/java tcp6 0 0 :::8080 :::* LISTEN 24715/java   . . ... View more

Re: Secure Webhdfs in Hadoop Hortonworks Cluster

by Master Mentor Master Mentor in Support Questions
‎01-30-2020 04:04 PM
‎01-30-2020 04:04 PM
@asmarz  As we see the error like: Failed to run the Ambari Server org.apache.ambari.server.AmbariException: Ambari Server Kerberos credentials check failed. Check KDC availability and JAAS configuration in /etc/ambari-server/conf/krb5JAASLogin.conf . 1. So can you please let us know how did you enable Kerberos for Ambari Server ?   or manually? 2. Do you have ambari-agent installed on the ambari server host?  and Do you have the Kerberos clients installed on the ambari server host? # yum info krb5-libs # yum info krb5-workstation 3. Do you have the correct KDC/AD address defined inside the file : # ps -ef | grep AmbariServer | grep --color krb5.conf # cat /etc/krb5.conf . 4. Are you able to do "kinit" to get a valid kerberos ticket using the same detail mentioned in the file "/etc/ambari-server/conf/krb5JAASLogin.conf" # kinit -kt /etc/security/ambariservername.keytab ambariservername@REALM.COM # klist .         ... View more
Contact Me
Community Statistics
Member Since ‎03-14-2016 01:07 PM
Last Visited
Posts 4,721
Kudos received 1096