About Shelton

Shelton · ‎02-08-2019

The below steps describe how to change the Namenode log level while logged on as hdfs with the below steps, without the need to restart the namenode Get the current log level $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange SubmittedLogName:BlockStateChange LogClass: org.apache.commons.logging.impl.Log4J LoggerEffectiveLevel: INFO Change to DEBUG $ hadoop daemonlog -setlevel {namenode_host}:50070BlockStateChange DEBUG Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=DEBUG SubmittedLogName:BlockStateChange LogClass: org.apache.commons.logging.impl.Log4J LoggerSubmittedLevel: DEBUG SettingLevel to DEBUG ... EffectiveLevel: DEBUG Validate DEBUG mode $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange SubmittedLogName:BlockStateChange LogClass: org.apache.commons.logging.impl.Log4J LoggerEffectiveLevel: DEBUG You should be able to notice the logging level in namenode.log has been updated, without restarting the service. After finishing your diagnostics you can reset the logging level back to INFO Reset to INFO $ hadoop daemonlog -setlevel {namenode_host}:50070BlockStateChange INFO Desired Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange&level=INFO SubmittedLogName:BlockStateChange LogClass: org.apache.commons.logging.impl.Log4J LoggerSubmittedLevel: INFO SettingLevel to INFO ... EffectiveLevel: INFO Validate INFO $ hadoop daemonlog -getlevel {namenode_host}:50070BlockStateChange Output Connecting to http://{namenode_host}:50070/logLevel?log=BlockStateChange SubmittedLogName:BlockStateChange LogClass: org.apache.commons.logging.impl.Log4J LoggerEffectiveLevel: INFO Happy hadooping !!!!

Shelton · ‎02-10-2019

@Michael Bronson HWX doesn't recommend upgrading an individual HDP component because one never knows the incompatibilities that could impact the other components and component selective upgrades tend to be a nightmare during a version upgrade The lastest HDP Kafka version is 11-2.1.x delivered by HDP 3.1 but ASF has its own rollout version and naming convention HTH

Shelton · ‎02-09-2019

@christophe VALMIR Usually, after the restart give the process a minute or 2 for the processes to pick up. Please don't forget to vote a helpful answer as the issue is resolved,that way other HCC users can quickly find the solution when they encounter the same issue. HTH

Shelton · ‎02-06-2019

@Chris Jenkins My pleasure I made your day and welcome to Big data space, having to go all through all this will make you better technically you've now seen the different facets to resolving a problem. Happy Hadooping !

Shelton · ‎02-07-2019

@Shraddha Singh Where machine is the FQDN and {rangerkms_password} is the rangerkms user password. The FQDN is the output of $hostname -f Re-run the below commands grant all privileges on rangerkms.* to 'rangerkms'@'machine' identified by '{rangerkms_password}'; grant all privileges on rangerkms.* to 'rangerkms'@'machine' with grant option; And let me know

tburke · ‎01-29-2019

Thanks Again! I do believe I found my issue. the repos where not complete and accurate on my ubuntu 18.04 builds , so I just copied repos from my xenial 16.04 box and replaced xenial with ubuntu then was able to install lafter update the kerberos client. here was my final repo for ubuntu 18.04 deb http://us.archive.ubuntu.com/ubuntu/ bionic main restricted deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates main restricted deb http://us.archive.ubuntu.com/ubuntu/ bionic universe deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates universe deb http://us.archive.ubuntu.com/ubuntu/ bionic multiverse deb http://us.archive.ubuntu.com/ubuntu/ bionic-updates multiverse deb http://us.archive.ubuntu.com/ubuntu/ bionic-backports main restricted universe multiverse deb http://security.ubuntu.com/ubuntu bionic-security main restricted deb http://security.ubuntu.com/ubuntu bionic-security universe deb http://security.ubuntu.com/ubuntu bionic-security multiverse

Shelton · ‎01-28-2019

@Michael Bronson If you have exhausted all other avenues YES, Step 1 Check and compare the /usr/hdp/current/kafka-broker symlinks Step 2 Download both env'es as backup from the problematic and functioning cluster Upload the functioning cluster env to the problematic one, since you have a backup Start kafka through ambari Step 3 sed -i 's/verify=platform_default/verify=disable/'/etc/python/cert-verification.cfg Step 4 Lastly, if the above steps don't remedy the issue, then remove and -re-install the ambari-agent and remember to manually point to the correct ambari server in the ambari-agent.ini

massoudm · ‎01-25-2019

Thanks Geoffrey. I copied the backup of the ambari.properties to expected location and ran the upgrade command again and it worked this time.

Shelton · ‎01-28-2019

@Bhushan Kandalkar Good it worked out but you shouldn't have omitted the information about the architecture ie Load balancer such info is critical in the analysis ....:-) Happy hadooping

Shelton · ‎01-24-2019

@Lokesh Mukku Good to know it has given you a better understanding. If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer. That would be a great help to Community users to find the solution quickly for these kinds of errors. Happy hadooping !!!!!

Online	Offline
Last Visited	‎03-29-2026 10:39 AM

Member Since	‎01-19-2017 04:35 AM
Last Visited	‎03-29-2026 10:39 AM
Posts	3,679
Kudos received	621

Cloudera Community

Re: Apache nifi memory consumption in kubernetes

Re: Nifi toolkit command for GitLabFlowRegistry

Re: Not able to delete the NiFi existing flow usin...

Re: Securing Nifi with SSL and using OIDC provider...

Re: External zookeeper and nifi cluster connection...

Set log level of namenode

Re: is it possible to upgrade only the kafka machi...

Re: hello, I try to follow the documentation. http...

Re: HDP 3.0.1 installed on Windows 10 / VMWare Wor...

Re: Error Ranger KMS server install on Ambari2.7.0...

Re: Kerberos with hortonworks stack 18.04 ubuntu

Re: cant start kafka broker from ambari

Re: ambari 2.7.1 to 2.7.3 upgrade failed

Re: Atlas Metadata server not starting

Re: namenode down?