Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

YARN HistoryServer Web Status issue

avatar
Contributor

Hello Team,

 

We recently upgraded the CM from version 7.2.X to 7.6.1. Since this is a production cluster we didn't do a restart of Cluster yet. However, we are getting alert with YARN HistoryServer. After checking the logs, we couldn't see any issue from the YARN HistoryServer but we found issues in the Cloudera agent. Please check the logs as shown below.

 

 

[16/Jun/2022 07:50:44 +0200] 3249 GM JOBHISTORY throttling_logger ERROR    (4 skipped) Error fetching metrics at 'https://xxx.xxx.com:19890/jmx'
Traceback (most recent call last):
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/generic/metric_collectors.py", line 223, in _collect_and_parse_and_return
    self._adapter.safety_valve))
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/util/url.py", line 305, in urlopen_with_retry_on_authentication_errors
    return function()
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/monitor/generic/metric_collectors.py", line 245, in _open_url
    cipher_list=self._tls_cipher_list)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/util/url.py", line 104, in urlopen_with_timeout
    return opener.open(url, data, timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 469, in error
    result = self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/urllib2_kerberos.py", line 203, in http_error_401
    retry = self.http_error_auth_reqed(host, req, headers)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/urllib2_kerberos.py", line 127, in http_error_auth_reqed
    return self.retry_http_kerberos_auth(req, headers, neg_value)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/urllib2_kerberos.py", line 143, in retry_http_kerberos_auth
    resp = self.parent.open(req)
  File "/usr/lib64/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/https.py", line 360, in http_error_default
    raise e
HTTPError: HTTP Error 403: Forbidden

 

 

Could you please share what can we do to fix the error?

 

Best Regards

Sayed Anisul Hoque

1 ACCEPTED SOLUTION

avatar
Contributor

Update: After the restart of the Cluster the issue went away. All good now.

View solution in original post

1 REPLY 1

avatar
Contributor

Update: After the restart of the Cluster the issue went away. All good now.