Member since
10-02-2017
116
Posts
3
Kudos Received
8
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1000 | 07-18-2020 12:04 PM | |
1728 | 09-11-2019 01:14 PM | |
2501 | 08-16-2019 08:17 AM | |
6348 | 08-15-2019 12:23 PM | |
4205 | 05-14-2019 08:48 AM |
12-22-2021
02:07 PM
Apache recently posted an update to the existing log4j2 vulnerabilities already discussed here. From the Apache page: "Apache Log4j2 versions 2.0-alpha1 through 2.16.0, excluding 2.12.3, did not protect from uncontrolled recursion from self-referential lookups. When the logging configuration uses a non-default Pattern Layout with a Context Lookup (for example, $${ctx:loginId}), attackers with control over Thread Context Map (MDC) input data can craft malicious input data that contains a recursive lookup, resulting in a StackOverflowError that will terminate the process. This is also known as a DOS (Denial of Service) attack." My company runs several private CDH6.3 clusters and we've already applied the original log4j2 patch from https://github.com/cloudera/cloudera-scripts-for-log4j. Can anyone confirm if CDH products are susceptible to CVE-2021-45105? Seems this particular vulnerability is enabled only if logging configuration uses a non-default pattern layout with a context lookup. The mitigation strategy differs from that taken by the existing log4j2 patch from Cloudera. Apache log4j security vulnerabilities: https://logging.apache.org/log4j/2.x/security.html#CVE-2021-45105
... View more
12-22-2021
07:56 AM
There was a recent update from Apache; https://logging.apache.org/log4j/2.x/security.html#CVE-2021-45105 "Apache Log4j2 does not always protect from infinite recursion in lookup evaluation", which seems to imply the mitigation strategy of removing references to jndilookup class will not address this. Is this a concern for CDH6.3? If so, will the patch script be updated to address?
... View more
02-05-2021
08:47 AM
Regarding the updated paywall policy - specifically the following comment: "Furthermore, when a license expires, users will no longer be able to access the Cloudera Manager Admin console to manage clusters until a valid license is uploaded. " We have several unlicensed clusters running. Do you interpret unlicensed clusters as clusters with expired licenses, in which case we stand to lose access to our Cloudera Manager consoles at any time now?
... View more
02-05-2021
06:45 AM
Unbelievable.
... View more
07-18-2020
12:04 PM
Solved: The Altus Director web console provides a way to update an environment's provider credentials, but only after all clusters/deployments have been deleted. To update the credentials for existing deployment, I had to use director's API: /api/d6.2/environments/{name}/provider/credentials "Update provider credentials for a specific environment"
... View more
07-18-2020
11:11 AM
During an overly ambitious cleanup effort, the IAM user / access key used by our Altus Director server was deleted from AWS and now we can no longer manage our clusters. I would like to either update the access key used by director, or remove the key and force director to rely on IAM role. Please advise.
... View more
09-11-2019
01:14 PM
We updated our Packer image build to skip the CIS rule for more restrictive umask (referenced above), after which Hue successfully started during cluster firstrun.
... View more
08-16-2019
09:42 AM
Some additional information... We use Packer to build our images, and apply RedHat's CIS security policy for compliance reasons which sets a more restrictive umask in /etc/bashrc if [ $UID -gt 199 ] && [ “/usr/bin/id -gn” = “/usr/bin/id -un” ]; then
umask 027
else
umask 022
fi I'm thinking this doesn't effect our CDH5 deployments because those images already have the correct package version installed, however during CDH6 bootstrap, updated version are required and are installed using 027 umask resulting in no permissions for non-root user. Does the cluster bootstrap process assume umask 022 for non-root users?
... View more
08-16-2019
09:06 AM
Using Cloudera Altus Director 6.3 to deploy CDH 6.3 to AWS. I submitted a similar post back in April, thinking that it had since been resolved - but apparently not. We are ramping up CDH6 deployments again since our initial testing in April and again are seeing Hue fail to start during first run of CDH6 deployments due to Python lib folders being too restrictive. Interestingly we do not see this issue when using the same CentOS 7.6 based AMI to deploy CDH 5.16.2 clusters. The Python lib permissions differ on between the CDH5 and CDH6 deployments: CDH5: ls -l /usr/lib/python2.7/site-packages/six* -rw-r--r--. 1 root root 29664 Jan 2 2015 /usr/lib/python2.7/site-packages/six.py -rw-r--r--. 1 root root 29708 Nov 20 2015 /usr/lib/python2.7/site-packages/six.pyc -rw-r--r--. 1 root root 29708 Nov 20 2015 /usr/lib/python2.7/site-packages/six.pyo /usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg-info: total 16 -rw-r--r--. 1 root root 1 Nov 20 2015 dependency_links.txt -rw-r--r--. 1 root root 1419 Nov 20 2015 PKG-INFO -rw-r--r--. 1 root root 249 Nov 20 2015 SOURCES.txt -rw-r--r--. 1 root root 4 Nov 20 2015 top_level.txt CDH6: ls -l /usr/lib/python2.7/site-packages/six* -rw-r-----. 1 root root 32452 Aug 15 18:09 /usr/lib/python2.7/site-packages/six.py -rw-r-----. 1 root root 31828 Aug 15 18:09 /usr/lib/python2.7/site-packages/six.pyc /usr/lib/python2.7/site-packages/six-1.12.0.dist-info: total 24 -rw-r-----. 1 root root 4 Aug 15 18:09 INSTALLER -rw-r-----. 1 root root 1066 Aug 15 18:09 LICENSE -rw-r-----. 1 root root 1940 Aug 15 18:09 METADATA -rw-r-----. 1 root root 537 Aug 15 18:09 RECORD -rw-r-----. 1 root root 4 Aug 15 18:09 top_level.txt -rw-r-----. 1 root root 110 Aug 15 18:09 WHEEL Again, we use the same AWS AMI for the CDH5 and 6 deployments. For CDH6 I have to manually fix the permissions on all the nodes prior to first run in order to avoid director declaring the deployment failed. This is really hampering our CDH6 rollout. + run_syncdb_and_migrate_subcommands + '[' 6 -ge 6 ']' + /opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue makemigrations --noinput Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue", line 9, in <module> from pkg_resources import load_entry_point File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module> @_call_aside File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside f(*args, **kwargs) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 574, in _build_master ws = cls() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 567, in __init__ self.add_entry(entry) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 623, in add_entry for dist in find_distributions(entry, True): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2056, in find_on_path for dist in factory(fullpath): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2118, in distributions_from_metadata if len(os.listdir(path)) == 0: OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/six-1.12.0.dist-info' + /opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue migrate --fake-initial Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue", line 9, in <module> from pkg_resources import load_entry_point File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module> @_call_aside File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside f(*args, **kwargs) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 574, in _build_master ws = cls() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 567, in __init__ self.add_entry(entry) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 623, in add_entry for dist in find_distributions(entry, True): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2056, in find_on_path for dist in factory(fullpath): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2118, in distributions_from_metadata if len(os.listdir(path)) == 0: OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/six-1.12.0.dist-info' + '[' dumpdata = runcpserver ']' + '[' syncdb = runcpserver ']' + '[' ldaptest = runcpserver ']' + exec /opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue runcpserver Traceback (most recent call last): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/bin/hue", line 9, in <module> from pkg_resources import load_entry_point File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in <module> @_call_aside File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside f(*args, **kwargs) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 574, in _build_master ws = cls() File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 567, in __init__ self.add_entry(entry) File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 623, in add_entry for dist in find_distributions(entry, True): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2056, in find_on_path for dist in factory(fullpath): File "/opt/cloudera/parcels/CDH-6.3.0-1.cdh6.3.0.p0.1279813/lib/hue/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2118, in distributions_from_metadata if len(os.listdir(path)) == 0: OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/six-1.12.0.dist-info’
... View more
Labels:
- Labels:
-
Cloudera Manager
08-16-2019
08:17 AM
We discovered that there were sporadic network issues in the tunnel between Azure and AWS ( our director instance is in AWS). Our assumption is that this was causing transient connection issues between director and Azure instances. Declaring issue solved for now as we no longer are experiencing it.
... View more