Member since
02-16-2017
21
Posts
1
Kudos Received
0
Solutions
02-11-2021
06:45 AM
the use case is for adding a host to the cluster. to download the packages, it just connects to local mirror (instead of archive.cloudera) this will also allow for an non-internet connect infrastructure.
... View more
02-10-2021
04:55 AM
am i allowed to publicly mirror archive.cloudera.com?
this is for Express version
... View more
Labels:
02-10-2021
03:33 AM
this might be an auth problem, i have noticed 404's as well. try to go to https://archive.cloudera.com/ and see you will be asked for login.
... View more
04-29-2020
02:42 AM
is there a place that shows the operating system requirements of future CDH versions? I am wondering specifically if there is a ubuntu focal version spec'd and what other OS's it supports?
... View more
Labels:
- Labels:
-
Cloudera Manager
10-24-2019
12:08 AM
thanks, i suppose one could just move that to wherever you want the home to be. this was part of a hack deployment, so ultimately not needed.
... View more
10-09-2019
06:26 AM
Spark2 2.3.0 on cdh.5.13 deployed with CSD and parcels. by default it uses /etc/spark2/conf, but it seems it should use /etc/spark2/conf.cloudera.spark2_on_yarn/ is there a way to set it properly? i could set it in spark2-env.sh but it seems not the appropriate place.
... View more
Labels:
- Labels:
-
Apache Spark
-
Cloudera Manager
06-20-2019
01:37 AM
thanks for your answer, do i need the .meta and .meta.tmp files?
... View more
06-19-2019
06:15 AM
so i have a flume agent with a spooldir channel that got too full (too many files error). the fix was to move the files out of the spooldir, move the checkpoint, and restart the agent, resulting in data flowing again. but now i have some flume-formatted data that i would like to ingest. can i move the files into the spooldir 10% at a time fx? would i have to read the files and add them to the flume source? additional notes: the reason for the spool dir was a missing sink (another flume agent- turned off)
... View more
Labels:
- Labels:
-
Apache Flume
11-09-2018
12:59 AM
i have a logging setup where i have a fluent/webhdfs host, for writing logs from fluent to hdfs. a problem is i can only write to the ACTIVE namenode, this sets some requirements to the logic in the fluent/webhdfs host that i would like to avoid. is there a way to make sure youre always posting to the ACTIVE namenode?
... View more
Labels:
- Labels:
-
HDFS
06-28-2018
06:06 AM
yes, but that requires cluster restart right? and also the copying on the journal node edit dirs.
... View more
06-28-2018
04:06 AM
thanks, this is what worked in the end, coupled with kill -9 for the really resilient procs.
... View more
06-28-2018
03:59 AM
so i want to move the Journalnode from 1 host to another. this is a cluster running cloudera manager 5.13 it's set up with Quorum Journal. currently there is 3 nodes: A,B,C I want to add node D. and then remove node C. why can i not do the following: 1. add journalnode role to D 2. remove journalnode role from C leaving me with A,B,D and a migrated journalnode.
... View more
Labels:
- Labels:
-
Cloudera Manager
04-25-2018
06:18 AM
UPDATE: did some experimentations on my own, and i deleted the pkg_resources, in /usr/lib/python2.7/dist-packages this apparantly fixed the issue as hue will now run, and allow me to log in. i am not sure if an update would also work...
... View more
04-24-2018
01:38 AM
upgraded from cdh 5.6 to 5.13
and since then, hue will not start.
stderr from running a "Start" on "Hue" through the Manager- GUI
++ tr '\n' :
++ find /usr/share/cmf/lib/plugins -maxdepth 1 -name '*.jar'
+ ADD_TO_CP=/usr/share/cmf/lib/plugins/tt-instrumentation-5.13.0.jar:/usr/share/cmf/lib/plugins/event-publish-5.13.0-shaded.jar:
+ [[ -n '' ]]
+ eval 'OLD_VALUE=$HADOOP_EXTRA_CLASSPATH_STRING'
++ OLD_VALUE=
+ NEW_VALUE=/usr/share/cmf/lib/plugins/tt-instrumentation-5.13.0.jar:/usr/share/cmf/lib/plugins/event-publish-5.13.0-shaded.jar:
+ export HADOOP_EXTRA_CLASSPATH_STRING=/usr/share/cmf/lib/plugins/tt-instrumentation-5.13.0.jar:/usr/share/cmf/lib/plugins/event-publish-5.13.0-shaded.jar
+ HADOOP_EXTRA_CLASSPATH_STRING=/usr/share/cmf/lib/plugins/tt-instrumentation-5.13.0.jar:/usr/share/cmf/lib/plugins/event-publish-5.13.0-shaded.jar
+ HUE=/opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue
+ [[ runcpserver == runcpserver ]]
+ grep -q '^\s*ssl_certificate\s*=\s*.\+' hue.ini hue_safety_valve.ini hue_safety_valve_server.ini
+ '[' '!' -z '' ']'
+ run_syncdb_and_migrate_subcommands
+ /opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue syncdb --noinput
+ '[' -z '' ']'
+ MERGE_FLAG=--merge
+ '[' 5 -ge 5 ']'
+ /opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue migrate --merge
+ '[' dumpdata = runcpserver ']'
+ '[' syncdb = runcpserver ']'
+ '[' ldaptest = runcpserver ']'
+ exec /opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue runcpserver
Traceback (most recent call last):
File "/opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue", line 8, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2707, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 686, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 584, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: distribute
it seems like it's a bad import in the hue script:
/opt/cloudera/parcels/CDH-5.13.0-1.cdh5.13.0.p0.29/lib/hue/build/env/bin/hue
#!/usr/bin/env python2.7
import os; activate_this=ocdn(os.path.dirname(os.path.realpath(__file__)), 'activate_this.py'); exec(compile(open(activate_this).read(), activate_this, 'exec'), dict(__file__=activate_this)); del os, activate_this
# EASY-INSTALL-ENTRY-SCRIPT: 'desktop==3.9.0','console_scripts','hue'
__requires__ = 'desktop==3.9.0'
import sys
from pkg_resources import load_entry_point
if __name__ == '__main__':
sys.exit(
load_entry_point('desktop==3.9.0', 'console_scripts', 'hue')()
)
should be changed to( changes on line 6 and 10)
#!/usr/bin/env python2.7
import os; activate_this=ocdn(os.path.dirname(os.path.realpath(__file__)), 'activate_this.py'); exec(compile(open(activate_this).read(), activate_this, 'exec'), dict(__file__=activate_this)); del os, activate_this
# EASY-INSTALL-ENTRY-SCRIPT: 'desktop==3.9.0','console_scripts','hue'
__requires__ = 'desktop==3.9.0'
import sys
import pkg_resources
if __name__ == '__main__':
sys.exit(
pkg_resources.EntryPoint.load('desktop==3.9.0', 'console_scripts', ‘hue’)()
)
is there another way to solve this?
something like putting the old pkg_resource into the path of the new?
... View more
Labels:
- Labels:
-
Cloudera Hue
-
Cloudera Manager
11-29-2017
05:30 AM
i am having problems forcing the cloudera/hadoop procs to use the new supervisord and assesing the impact of this not working. i am upgrading from cdh5.6 to cdh5.13 did upgrade all the cloudera manager daemons and agents to 5.13 some nodes are not using the new supervisord, this is seen from the text below, taken from the host inspector, it gives 3 groups using 3 different supervisord group1: Supervisord 3.0-cm5.13.0 group2L Supervisord 3.0-cm5.6.0 group3: Supervisord 3.0 i can move hosts to use the new supervisord, by killing the old process, kill $(cat /run/cloudera-scm-agent/supervisord/supervisord.pid) this works, something starts a new, korrekt version, of supervisord. this seems like a not-nice way to do it, and causes some warning in the manager. is there another way to do this? also, what is the potential risks in not using the same supervisord version, will the nodes be unable to communicate? can i upgrade parcels?
... View more
Labels:
- Labels:
-
Cloudera Manager