Skip to main content
dturner
dturner
Gold ⭐️
April 17, 2019
Solved

Permission errors loading Python modules: CDH6.2, CentOS7.6 FirstRun

I've read the known issues for Hue in the Altus Director 6.2 docs, however according to the information in the docs, this shouldn't be applicable to CM6.2 and CentOS 7.6.  I'm seeing the following error when Hue attempts to start during the cluster's firstrun phase:

 

/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/bin/hue makemigrations --noinput
Traceback (most recent call last):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/bin/hue", line 9, in <module>
from pkg_resources import load_entry_point
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3191, in <module>
@_call_aside
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3175, in _call_aside
f(*args, **kwargs)
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3204, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 574, in _build_master
ws = cls()
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 567, in __init__
self.add_entry(entry)
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 623, in add_entry
for dist in find_distributions(entry, True):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2033, in find_on_path
for dist in factory(fullpath):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2095, in distributions_from_metadata
if len(os.listdir(path)) == 0:
OSError: [Errno 13] Permission denied: '/usr/lib64/python2.7/site-packages/wrapt-1.11.1.dist-info'
+ /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/bin/hue migrate --fake-initial
Traceback (most recent call last):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/bin/hue", line 9, in <module>
from pkg_resources import load_entry_point
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3191, in <module>
@_call_aside
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3175, in _call_aside
f(*args, **kwargs)
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3204, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 574, in _build_master
ws = cls()
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 567, in __init__
self.add_entry(entry)
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 623, in add_entry
for dist in find_distributions(entry, True):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2033, in find_on_path
for dist in factory(fullpath):
File "/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2095, in distributions_from_metadata
if len(os.listdir(path)) == 0:
OSError: [Errno 13] Permission denied: '/usr/lib64/python2.7/site-packages/wrapt-1.11.1.dist-info'

 

 

Update:

 

I "solved" this by updating the site-package permissions under /usr/lib /usr/lib64, as they were locked down to root.  If I were to guess, some modules were pip installed without the -u switch, locking down access to root only.  I assume this should be handled by the deployment scripts when preparing the cluster nodes.

 

dturner
dturnerAnswer
Gold ⭐️
May 14, 2019

Update:  I rebuilt our image and no longer see this issue.  I still don't know why we didn't see this issue with Altus 2.8 using the same image, but I'll mark as resolved since we no longer see it with the new image.

View in thread

5 replies

dturner
dturnerAuthor
Gold ⭐️
April 18, 2019

I tried another deployment and hit the same errors.  All other cluster services start fine during first run.  Hue continues to fail to start due to insufficient permissions to load Python modules.  Pretty sure I could fix this if someone pointed me to where you handle the install of these Python modules.  Suprised no one else is reporting it.  

Translate
aarman
Gold ⭐️
aarman
Gold ⭐️
April 19, 2019

Hi dturner,

 

I just attempted this and wasn't able to reproduce the issue. My cluster creation which included Hue succeeded using CM+CDH 6.2 with CentOS7.6.

 

In fact I couldn't even find the offending package (wrapt) anywhere on the Hue instance, so there's something different in our environments/installation.

 

Maybe more information about the enviornment/configuration would help narrow down the problem (whether your using preloaded/prextracted images, AWS or Azure, any special bootstrap scripts, etc)

Translate
dturner
dturnerAuthor
Gold ⭐️
April 19, 2019

Thank you for your response.

 

We base our image on the stock CentOS AMI provided by AWS, with some security policies and tuning applied. Interestingly enough, we've deployed several 5.16.1 clusters using the same AMI.  This appears to be specific to CDH6.2.   I encounter this issue whether or not I try to include Hue in the Altus cluster config, or add the service post deployment from Cloudera Manager.   Wrapt is just the first module it complains about.  There are several others.  I ended up simply changing directory / file permissions for all dist-info, afterwhich I could add Hue service. 

 

 

chmod 755 /usr/lib64/python2.7/site-packages/*.dist-info
chmod -R 644 /usr/lib64/python2.7/site-packages/*.dist-info/
chmod 755 /usr/lib/python2.7/site-packages/*.dist-info
chmod -R 644 /usr/lib/python2.7/site-packages/*.dist-info/

 

I am using the latest version of Altus Director (6.2).  

Translate
Terms & ConditionsAccessibility statement