About bgooley

bgooley · ‎06-24-2019

@VijayM , It appears that you collected the "test-db-connection" command for Navigator which succeeded. Based on the Cloudera command, the Navigator Metadata server role is on a301-8744-7259.ldn.swissbank.com, so make sure that you were looking in /var/run/cloudera-scm-agent/process for the METASERVER process. If you don't see any, then it is likely the agent will tell us more of the story of what happened. In general, this is the series of events that happens when you attempt to start a role via Cloudera Manager: Cloudera Manager sends a message to the agent on port 9000 asking it to heartbeat immediately. The agent sends a heartbeat to Cloudera Manager on port 7182 Cloudera Manager compiles a heartbeat response including any files that need to be used to start and run the process The agent lays out the files in a new process directory The agent signals the supervisord process to start the role with a specified shell script The supervisor executes the startup shell script to start the role Since it appears you do not have any process directories, there must be a problem before the agent even lays down the process configuration files. If that is the case, the agent log, on the host where the Metadata Server role is configured to run should have some clues. Please share /var/log/cloudera-scm-agent/cloudera-scm-agent.log and let us know the estimated time you tried to start the Metadata Server.

bgooley · ‎06-21-2019

@urbanlad20 , That stack reminds me of HDFS-12369 but I also thought it should be fixed in CDH 5.12.2. I think it would be good to have HDFS folks look at this; I'll move thread to HDFS.

bgooley · ‎06-18-2019

@MichalAR , Right, so SAML can be used for Authentication and then LDAP for user/group sync. If you are not using LDAP for authentication, then "Create LDAP users on login" won't impact you. If you want to prevent the creation of a Hue user for a new user login, you can set the following: [libsaml] create_users_on_login=False If you do that, though, you need to be sure that you have all of your users already in Hue before they authenticate; otherwise, they will get an error. If you would like to leave create_users_on_login "True" but change the default group membership, you can adjust the "default" group that is set for new users. To do so, set: [useradmin] default_user_group=<name_of_your_preferred_group> That way, you don't prevent users from authenticating via SAML if they don't already exist as Hue users, but you can restrict the resources they can access. It's just another thing to consider that may help you achieve the type of configuration you want.

bgooley · ‎06-17-2019

Hi @MichalAR , Hue does not support group mapping based on SAML attributes at this time. For now, the general workaround is to use LDAP sync if possible to automate user group membership.

bgooley · ‎06-07-2019

@Kevin_Z , Current releases of Cloudera Manager do not support Kerberos authentication for access to the CM UI and API. We have added that feature, though, and it is targeted for future releases.

bgooley · ‎06-03-2019

Hi @Ben37 , Welcome to the Cloudera Community! Cloudera Manager monitors host resources and stores those in the Host Monitor. Information is stored via leveldb, so it is not retained in a "log" file as you seeking. You can find information about how to build charts and query Host and Service Monitor data here: https://www.cloudera.com/documentation/enterprise/6/6.2/topics/cm_dg_chart_time_series_data.html The intent of the Host resource metrics provided by Cloudera Manager are intended for use within the context of the cluster itself to monitor the ability of the host to service the needs of the hadoop services. The information is available via charts in your host, service, and role dashboards. If you are using Cloudera Manager in your cluster, then it will help to have a look through some of the information here to understand what is available in terms of monitoring cluster health (including host resources): https://www.cloudera.com/documentation/enterprise/6/6.2/topics/cm_dg_about.html Cheers, Ben

bgooley · ‎05-22-2019

@rdbb , It is possible to even have a one-node cluster (With CM and CDH), but that is certainly not something that you would want to do if you want to leverage the inherent redundancy and load distrobution of CDH. Functionally, you can put roles most anywhere you please; as far as what is best for your desired use cases, though, we can't really comment without more context. If you only have 4 hosts, you still need to consider what sort of memory the roles that are on those hosts will demand (as well as CPU and disk). Also, do you want to use High Availability for HDFS and YARN (NameNode and Resource Manager)? Given enough RAM, disk, etc. you could put master host and edge node on one node and then use the other 3 as worker nodes. It is more about what you want to get out of this cluster, how important uptime is, and what resources you have. If you don't care too much and just want to play around, the configuration I described with master/edge on one host and workers on the other 3 is fine. In fact, that's basically what we do for our nightly builds here at Cloudera. I hope that halps a bit. Feel free to ask more questions.

bgooley · ‎04-30-2019

Hi @banshidhar_saho , Since the stack trace shows RunJar.java being used, that indicates the Java option you need is: java.io.tmpdir If you can se that in your "Java Configuration" safety valves for Yarn that should help. Since we don't see the whole stack trace in your post, we can't tell which safety valve would apply to that situation exactly.

bgooley · ‎04-23-2019

@Bildervic, "SIGBUS (0x7)" can mean a few things, but one of the most common ones is that a directory that Java needs to use is full (no more free disk space). The fact that your Node Manager was running and then failed and then failed to start supports that type of possible cause. Since the crash is in libleveldbjni, that gives us more evidence that a directory may be full since that indicates Java was accessing local files (on disk). I would suggest checking disk space on all volumes on that host. If there a volume that is full, then try freeing up some space and start the Nodemanager again.

bgooley · ‎03-22-2019

@IVenkatesh, Cloudera Manager knows to which database server to connect based on what it sees in /etc/cloudera-scm-server/db.properties We would not know which server should be used. It sounds like you'll have to figure out what has changed since you tried "moving" mysql server. If you used to have more users, try accessing each 'scm' database with a mysql client and see what rows exist in the USERS table. That might give you some clues.

Online	Offline
Last Visited	‎04-24-2020 01:13 PM

Member Since	‎04-22-2014 02:47 PM
Last Visited	‎04-24-2020 01:13 PM
Posts	1,218
Kudos received	340

Cloudera Community

Re: ALL hadoop-mapreduce-examples.jar fail cdh6

Re: YARN NodeManagers failed to start with permiss...

Re: Kerberos not authenticating from Hadoop Gatewa...

Re: Sqoop connection to Kerberos authenticated RDB...

Re: Failed to format namenode error during while i...

Re: Cloudera Navigator Metadata Server not startin...

Re: Failed to start namenode. java.io.FileNotFound...

Re: Hue SAML group mapping

Re: Hue SAML group mapping

Re: logging in to cloudera manager Admin Console w...

Re: need your help about resource logs

Re: Is it possible to run manager, utility and edg...

Re: Mapreduce job failure due to hadoop-unjarxxxxx...

Re: Failed to start role -YARN- NodeManager (n...

Re: AuthenticationFailureEventListener: Authentica...