Cloudera Community
apappu
user rank
user rank
Guru

Re: how to rotate gc files for ambari services

by Guru Guru in Support Questions
‎08-13-2018 10:36 PM
‎08-13-2018 10:36 PM
@Michael Bronson For Kafka its not even enabled GC logging I guess - hence you can ignore that part. but you can set that like below. In kafka-env section, From export KAFKA_KERBEROS_PARAMS="-Djavax.security.auth.useSubjectCredsOnly=false to: export KAFKA_KERBEROS_PARAMS="-Djavax.security.auth.useSubjectCredsOnly=false -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M If you satisfied with answers then please mark my comment as correct answer. this will help others as well. ... View more

Re: how to rotate gc files for ambari services

by Guru Guru in Support Questions
‎08-13-2018 10:03 PM
‎08-13-2018 10:03 PM
@Michael Bronson Just adding KAFKA_GC_LOG_OPTS="-XX:+UseGCLogFileRotation-XX:NumberOfGCLogFiles=5-XX:GCLogFileSize=2M" will not be sufficient - will have to use that variable during the service start. Regarding HDFS: In hadoop-env template wherever you find -XX:+PrintGCDetails parameter you can add -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M For example: From: HADOOP_JOBTRACKER_OPTS="-server -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:ErrorFile={{hdfs_log_dir_prefix}}/$USER/hs_err_pid%p.log -XX:NewSize={{jtnode_opt_newsize}} -XX:MaxNewSize={{jtnode_opt_maxnewsize}} -Xloggc:{{hdfs_log_dir_prefix}}/$USER/gc.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xmx{{jtnode_heapsize}} -Dhadoop.security.logger=INFO,DRFAS -Dmapred.audit.logger=INFO,MRAUDIT -Dhadoop.mapreduce.jobsummary.logger=INFO,JSA ${HADOOP_JOBTRACKER_OPTS}" To: HADOOP_JOBTRACKER_OPTS="-server -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:ErrorFile={{hdfs_log_dir_prefix}}/$USER/hs_err_pid%p.log -XX:NewSize={{jtnode_opt_newsize}} -XX:MaxNewSize={{jtnode_opt_maxnewsize}} -Xloggc:{{hdfs_log_dir_prefix}}/$USER/gc.log-`date +'%Y%m%d%H%M'` -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Xmx{{jtnode_heapsize}} -Dhadoop.security.logger=INFO,DRFAS -Dmapred.audit.logger=INFO,MRAUDIT -Dhadoop.mapreduce.jobsummary.logger=INFO,JSA ${HADOOP_JOBTRACKER_OPTS}" ... View more

Re: how to rotate gc files for ambari services

by Guru Guru in Support Questions
‎08-13-2018 08:44 PM
‎08-13-2018 08:44 PM
@Michael Bronson For HDFS to GC there would be below config/property -XX:+PrintGCDetails -XX:+PrintGCTimeStamps Next to that you can append like below -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=2M Note: you need to have space between each parameter. For Kafka - can you tell me how did you enable the GC logging? by default it may not have been enabled - ... View more

Re: how to rotate gc files for ambari services

by Guru Guru in Support Questions
‎08-13-2018 07:19 PM
1 Kudo
‎08-13-2018 07:19 PM
1 Kudo
@Michael Bronson You can follow same approach - those are JVM parameters and it should work for any service. you can add these parameters in env template in ambari for any of those services. -XX:+UseGCLogFileRotation-XX:NumberOfGCLogFiles=5-XX:GCLogFileSize=2M ... View more

Re: Timeline Service V2.0 Reader not starting

by Guru Guru in Support Questions
‎08-11-2018 01:11 AM
‎08-11-2018 01:11 AM
@Daniel Zafar yes - that is correct, its correct path. May be you can attempt to execute manually and see what happens. 1. cd /var/lib/ambari-agent 2. ./ambari-sudo.sh su yarn-ats -l -s /bin/bash -c 'export PATH='"'"'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/texlive/2016/bin/x86_64-linux:/usr/lib64/qt-3.3/bin:/usr/local/texlive/2016/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/maven/bin:/root/bin:/opt/maven/bin:/opt/maven/bin:/var/lib/ambari-agent'"'"' ; sleep 10;export HBASE_CLASSPATH_PREFIX=/usr/hdp/3.0.0.0-1634/hadoop-yarn/timelineservice/*; /usr/hdp/3.0.0.0-1634/hbase/bin/hbase --config /usr/hdp/3.0.0.0-1634/hadoop/conf/embedded-yarn-ats-hbase org.apache.hadoop.yarn.server.timelineservice.storage.TimelineSchemaCreator -Dhbase.client.retries.number=35 -create -s' This would print some logging in the UI ... View more

Re: Timeline Service V2.0 Reader not starting

by Guru Guru in Support Questions
‎08-11-2018 12:19 AM
‎08-11-2018 12:19 AM
@Daniel Zafar It looks like service restart is taking much longer time and Ambari is timing out - can you timeline service logs to understand why it is taking that long? ... View more

Re: KNOX SSO ambari login redirect Issue

by Guru Guru in Support Questions
‎08-09-2018 09:05 PM
‎08-09-2018 09:05 PM
@soumya swain I have created article for Ambari SSO - you can review and check that to see if that helps. https://community.hortonworks.com/articles/212005/steps-to-enable-knox-sso-for-ambari-1.html ... View more

Steps to enable Knox SSO for Ambari

by Guru Guru in Community Articles
‎08-09-2018 08:37 PM
2 Kudos
‎08-09-2018 08:37 PM
2 Kudos
Pre-requisite: 1. Setup Ambari with LDAP and Sync. 2. Setup Knox and point to same LDAP as Ambari server. Enable SSO for Ambari: 1. Get the Knox public cert by running below. openssl s_client -connect KNOXHOST:8443 <<<'' | openssl x509 -out /tmp/knox.crt 2. Run "ambari-server setup-sso" 3. "provider URL": Enter https://<hostname>:8443/gateway/knoxsso/api/v1/websso 4. "Public Certificate pem" : Provide step1 cert file content without BEGIN/END blocks. -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- Note: Make sure your /etc/ambari-server/conf/jwt-cert.pem file should have only one BEGIN/END 5. You can select default for rest of the configs. 6. Re-start Ambari server: Knox Configurations 1. If Ambari and Knox is in different host then Whitelist Ambari URL. In Advanced knoxsso-topology modify below config for whitelisting all (or you can write regex for specific) <param> <name>knoxsso.redirect.whitelist.regex</name> <value>.*</value> </param> 2. Re-start Knox server. Now try accessing Ambari using http://HOSTNAME/IP:PORT/ 1. It should re-direct to the Knox page 2. Enter the username/password and submit 3. It will take back to Ambari page and logged in. For any issues refer /var/log/knox/gateway.log and /var/log/ambari-server/ambari-server.log files to get some clue on failures. ... View more

Re: YARN: cannot start App Timeline Server

by Guru Guru in Support Questions
‎08-09-2018 07:46 PM
‎08-09-2018 07:46 PM
@Victor L looks like your namenode has issues -can you restart namenode and then attempt again to see if that helps? ... View more

Re: Unable to start the Name Node

by Guru Guru in Support Questions
‎08-09-2018 07:18 PM
‎08-09-2018 07:18 PM
@Sambasivam Subramanian Can you please post that error stack in code block? Also check namenode logs to see if there is any errors. ... View more
Contact Me
Online
Offline
Last Visited
‎07-24-2021 08:27 PM
Community Statistics
Member Since ‎06-20-2016 09:43 PM
Last Visited ‎07-24-2021 08:27 PM
Posts 308
Kudos received 103