About lwang

lwang · ‎06-04-2019

Hi @Cloudman , Welcome to Cloudera community! May I know what CM API version you are using? I feel below maybe the doc you are looking for: CM 5.8.x: https://cloudera.github.io/cm_api/apidocs/v13/path__clusters_-clusterName-_services_-serviceName-_replications_-scheduleId-_history.html CM 6.2.x: https://archive.cloudera.com/cm6/6.2.0/generic/jar/cm_api/apidocs/resource_ReplicationsResource.html#resource_ReplicationsResource_ClustersResourceV32_ServicesResourceV32_ReplicationsResourceV32_readHistory_GET Hope this helps, Li

lwang · ‎05-24-2019

Hi @wert_1311 , Thanks for posting the logs. As we can see from the JVM pause messages in the log snippet: 5:48:49.569 AM INFO JvmPauseMonitor Detected pause in JVM or host machine (e.g. a stop the world GC, or JVM not scheduled): paused approximately 2448ms: GC pool 'ParNew' had collection(s): count=1 time=0ms, GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=2534ms Depends on how big your cluster is, you may need to increase the memory assigned to Service Monitor (SMON) role. Please see the documentation talking about the HMON and SMON memory configuration: https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_storage.html#concept_ixl_hrk_nn Also, you can think of tuning garbage collection by enabling G1GC for SMON: From Cloudera Manager UI > Cloudera Management Services > Configuration > under SCOPE select “Service Monitor” > under CATEGORY select Advanced > in the "Java Configuration Options for Service Monitor" set the following: -XX:+UseG1GC -XX:-UseConcMarkSweepGC -XX:-UseParNewGC Then restart SMON. For more information about tuning G1, see the Oracle documentation on tuning garbage collection: https://docs.oracle.com/cd/E40972_01/doc.70/e40973/cnf_jvmgc.htm#autoId0 Thanks and hope this helps, Li

lwang · ‎05-10-2019

Hi @Ryanp , You are very welcome and glad to hear the issue is resolved! Thanks, Li

lwang · ‎05-09-2019

Hi @Ryanp , Here are the snippet from the knowledge article: A workaround can be achieved using the steps below. (NMS=Navigator Metadata Server) 1. Identify IP address of the NMS. (can be gathered in Cloudera Management services->instances->Navigator Metadata Server) 2. Add the following line to the bottom of /etc/hosts on the NMS 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 <NMS_IP_ADDRESS> java.sun.com 3. On NMS, install httpd, depending on your Linux distribution. As this host does not have internet access, you will need to manually import and install rpms. https://httpd.apache.org/docs/2.4/platform/rpm.html 4. Make a directory for java dtd file, and adjust permissions. mkdir /var/www/html/dtd; chown 755 /var/www/html/dtd 5. Acquire Java DTD file from the following URL. http://java.sun.com/dtd/web-app_2_3.dtd 6. Copy the file to /var/www/html/dtd directory on the NMS. Ensure the file is readable by "other". 7. Enable httpd server sudo systemctl enable httpd; sudo systemctl start httpd 8. Start NMS. NOTE: httpd must run for each NMS restart, this is not a one-off workaround. Systemctl command should ensure that this is the case. If the above workaround is not possible, a patch is also available from Cloudera Support until this is resolved in an upcoming release of Cloudera Manager. Thanks and hope this helps, Li

lwang · ‎05-09-2019

Hi @Ryanp , Does your cluster happen to be isolated from Internet? If so, then you may hit a known issue: In Cloudera Manager 6.2.0, if the Navigator Metadata server host doesn't have access to the internet, the Metadata data server role will fail to start. We have a knowledge article on this topic and it outlines the workaround steps to fix the issue: https://community.cloudera.com/t5/Customer/Navigator-Metadata-Server-Doesn-t-start-on-isolated-cluster/ta-p/89479 Thanks, Li

lwang · ‎03-25-2019

Hi @MantuDeka , Yes, if event server is not working, you won't be able to receive alerts. The alerts basically are the events which are marked as alerts based on CM configuration checking. About Report Manager, we need to check the RM process logs and role log for more details about what went wrong. By default: 1) On the Report Manager host, find out the process logs: /var/run/cloudera-scm-agent/process/<process-ID>-cloudera-mgmt-REPORTSMANAGER/logs/stderr and stdout.log 2) On the same host, find out the role log: /var/log/cloudera-scm-headlamp/mgmt-cmf-mgmt-REPORTSMANAGER-<hostname>.log.out Thanks and hope it helps, Li

lwang · ‎03-20-2019

Hi @MantuDeka , Thanks for providing the log file. The error message indicates that the Event Server index has an corruption. However, the log snippet does not tell what caused it. For resolution, there is an option to attempt to fix the index which involves some steps, however the index may not be recoverable. Alternatively a quick fix would be to start the Event Server with a fresh new index. This will not cause any impact to cluster operation, but you will not be able to search for previous events in Cloudera Manager UI, CM -> Diagnostics -> Events page. Here are the steps for quick fix: 1. Stop the Event Server role instance in CM 2. Backup the current data directory (you can find the value from CM UI and search for -> "Event Server Index Directory"), default value usually is /var/lib/cloudera-scm-eventserver/ (just for safety reasons, this backup is likely not needed afterwards) 3. Empty the data directory: # rm -rf /var/lib/cloudera-scm-eventserver/* 4. Start the Event Server role instance in CM 5. Monitor the Event Server role logs in /var/log/cloudera-scm-eventserver/ directory, it should confirm the process is able to start up and operate Thanks, Li

lwang · ‎03-19-2019

Hi @MantuDeka , To second @gzigldrum 's feedback, you can find out more information from some logs. By default: 1) On the Event Server host, find out the process logs: /var/run/cloudera-scm-agent/process/<process-ID>-cloudera-mgmt-EVENTSERVER/logs/stderr and stdout.log 2) On the same host, find out the role log: /var/log/cloudera-scm-eventserver/mgmt-cmf-mgmt-EVENTSERVER-<hostname>.log.out Thanks and hope it helps, Li

lwang · ‎03-11-2019

Hi @krieger , You are very welcome and glad to hear the issue got resolved! Cheers, Li

lwang · ‎03-08-2019

Hi @krieger , Could you please run this command and see if it returns any result? # grep '^[[:blank:]]' /etc/cloudera-scm-agent/config.ini Above command can rule out if there are any leading spaces in the /etc/cloudera-scm-agent/config.ini. See below example. The example showing that there's a leading space in front of server_port=7182. This will cause the config parser to fail. # grep '^[[:blank:]]' /etc/cloudera-scm-agent/config.ini server_port=7182 From the content you pasted, I feel there is a leading space before the line "use_tls=0". If the command does return the line, could you please remove the leading space in the line and then try to restart agent again? Thanks and hope this helps, Li

Online	Offline
Last Visited	‎05-05-2020 02:22 PM

Member Since	‎05-27-2014 10:28 AM
Last Visited	‎05-05-2020 02:22 PM
Posts	1,499
Kudos received	75

Cloudera Community

Re: Error when Preparing the Cloudera Manager Serv...

Re: Authentication methods for Cloudera Express 6....

Re: HUE UI cache's username and pasword

Re: Solr through Hue is throwing 403 request excep...

Re: cloudera scm server start issues - java

Re: Retrieve Command Details from cm_client

Re: Service Monitor restarts repeatedly

Re: Cloudera Navigatror status 503

Re: Cloudera Navigatror status 503

Re: Cloudera Navigatror status 503

Re: Event Server is failed to start.

Re: Event Server is failed to start.

Re: Event Server is failed to start.

Re: Host Monitor Fails To Start on Host After Spar...

Re: Host Monitor Fails To Start on Host After Spar...