Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
New Contributor

SYMPTOMS:

HDP running has recently been upgraded to Isilon 8.0.1.0 and kerberized.

When trying to start services through Ambari, user receives the following stderr:

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 289, in <module>
    Resourcemanager().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 124, in start
    self.wait_for_dfs_directories_created(params.entity_groupfs_store_dir, params.entity_groupfs_active_dir)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 246, in wait_for_dfs_directories_created
    self.wait_for_dfs_directory_created(dir_path, ignored_dfs_dirs)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/decorator.py", line 55, in wrapper
    return function(*args, **kwargs)
  File "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py", line 268, in wait_for_dfs_directory_created
    list_status = util.run_command(dir_path, 'GETFILESTATUS', method='GET', ignore_status_codes=['404'], assertable_result=False)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py", line 192, in run_command
    raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'curl -sS -L -w '%{http_code}' -X GET --negotiate -u : 'http://hwx.isilon.support:8082/webhdfs/v1/hwx/done/?op=GETFILESTATUS&user.name=hdfs'' returned status_code=401. 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>

Services are not able to authenticate.

ROOT CAUSE: Isilon bug number: 84897

RESOLUTION: During Isilon 8.0.1.0 upgrade, permissions are incorrectly changed on /etc/krb5.conf to 660 (instead of 644). This causes WebHDFS to be unable to check the kerberos tickets. This issue is fixed in Isilon 8.0.1.1. The workaround was to change the permissions on /etc/krb5.conf to 644. Please reach out to Dell/EMC if you need assistance with the workaround.

379 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎12-29-2016 12:59 AM
Updated by:
 
Contributors
Top Kudoed Authors