hi in the upgrade documentation the following is stated: "Upgrades from Cloudera Enterprise 5.15 to 6.0.0 Not supported" does it mean specifically to cdh 6.0.0 version or to any 6.x version? thanks Doron ... View more

hi We've got the following error: org.apache.openjpa.persistence.PersistenceException: ERROR: value too long for type character varying(255) {prepstmnt 190305649 INSERT INTO WF_JOBS (id, app_name, app_path, conf, created_time, end_time, external_id, group_name, last_modified_time, log_token, parent_id, proto_action_conf, run, sla_xml, start_time, status, user_name, wf_instance) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=0, state=22001] and to solve it we've altered the WF_JOBS table, resizing the app_path and app_name to be varchar(1024). Is it acceptable to modify the oozie database? Will it cause us problems in other parts of oozie? We did while using postgreSQL for oozie db. Thanks Doron ... View more

In our testing environments we're aleays short on space... So we thought that we can remove the /opt/cloudera/parcel-cache and /opt/cloudera/parcel-repo - after all the parcel is already installed and distributed. Big mistake... i failed to start the cloudera manager services and got error like: ========= [29/Jun/2015 10:57:41 +0000] 2917 MainThread util INFO Using generic parser for process cloudera-mgmt-HOSTMONITOR [29/Jun/2015 10:57:41 +0000] 2917 MainThread agent INFO Activating Process 1019-cloudera-mgmt-HOSTMONITOR [29/Jun/2015 10:57:41 +0000] 2917 MainThread agent INFO Created /var/run/cloudera-scm-agent/process/1019-cloudera-mgmt-HOSTMONITOR [29/Jun/2015 10:57:41 +0000] 2917 MainThread agent INFO Chowning /var/run/cloudera-scm-agent/process/1019-cloudera-mgmt-HOSTMONITOR to cloudera-scm (781) cloudera-scm (501) [29/Jun/2015 10:57:41 +0000] 2917 MainThread agent INFO Chmod'ing /var/run/cloudera-scm-agent/process/1019-cloudera-mgmt-HOSTMONITOR to 0751 [29/Jun/2015 10:57:41 +0000] 2917 MainThread agent ERROR Failed to activate {u'refresh_files': [u'cloudera-stack-monitor.properties'], u'config_generation': 0, u'auto_restart': True, u'running': True, u'required_tags': [], u'user': u'cloudera-scm', u'special_file_info': [], u'group': u'cloudera-scm', u'id': 1019, u'status_links': {u'status': u'http://ilmtxbda146.corp.amdocs.com:8091/'}, u'name': u'cloudera-mgmt-HOSTMONITOR', u'run_generation': 1, u'environment': {u'MGMT_LOG_FILE': u'mgmt-cmf-mgmt-HOSTMONITOR-ilmtxbda146.corp.amdocs.com.log.out', u'CDH_VERSION': u'-1', u'FIREHOSE_JAVA_OPTS': u'-Xms1073741824 -Xmx1073741824 -XX:OnOutOfMemoryError={{AGENT_COMMON_DIR}}/killparent.sh'}, u'optional_tags': [], u'program': u'mgmt/mgmt.sh', u'arguments': [u'firehose', u'--pipeline-type', u'HOST_MONITORING'], u'parcels': {u'CDH': u'5.3.0-1.cdh5.3.0.p0.30'}, u'resources': [{u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': {u'limit_memlock': None, u'limit_fds': None}, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/cloudera-scm-firehose', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'cloudera-scm', u'user': u'cloudera-scm', u'mode': 509}, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'10.233.164.119', u'port': 8091}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'10.233.164.119', u'port': 9995}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'10.233.164.119', u'port': 9994}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/lib/cloudera-host-monitor', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'cloudera-scm', u'user': u'cloudera-scm', u'mode': 509}, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': True, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/cloudera-scm-firehose/stacks', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'cloudera-scm', u'user': u'cloudera-scm', u'mode': 493}, u'cpu': None}], u'one_off': False} Traceback (most recent call last): File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 1234, in handle_heartbeat_response new_process.activate() File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 2438, in activate self.update_process_environment_and_parcels() File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 2601, in update_process_environment_and_parcels parcels_in_use = self.agent.repo.prepare_environment(self.raw['parcels'], AttributeError: 'NoneType' object has no attribute 'prepare_environment' [29/Jun/2015 10:57:57 +0000] 2917 CP Server Thread-12 _cplogging INFO 10.233.164.119 - - [29/Jun/2015:10:57:57] "GET /download_log?path=%2Fvar%2Flog%2Fcloudera-scm-firehose%2Fmgmt-cmf-mgmt-HOSTMONITOR-ilmtxbda146.corp.amdocs.com.log.out&onlyTail=true&compress=false HTTP/1.1" 200 2011 "" "Java/1.7.0_10" [29/Jun/2015 11:07:28 +0000] 2917 MonitorDaemon-Reporter throttling_logger ERROR (9 skipped) Error sending messages to firehose: mgmt-HOSTMONITOR-14f3c3b6446d19e4dcc4dab16355bb71 Traceback (most recent call last): File "/usr/lib64/cmf/agent/src/cmf/monitor/firehose.py", line 70, in _send self._port) File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/avro-1.6.3-py2.6.egg/avro/ipc.py", line 464, in __init__ self.conn.connect() File "/usr/lib64/python2.6/httplib.py", line 720, in connect self.timeout) File "/usr/lib64/python2.6/socket.py", line 567, in create_connection raise error, msg error: [Errno 111] Connection refused =========   Luckily for me, I had the same parcel on a different environment so I could copy it and environment was happy again.   My questions for cloudera people: * Why do I need the parcels after it was successfully installed? (all the jars and exe are available from /opt/cloudera/parcels/CDH) * Do you have any suggestions for saving space? - again this is for small testing environments and not for production.   Thanks Doron ... View more

Can I update oozie-site.xml using REST api?   We added our own external function to oozie. After each installation we need to go manually and change things in oozie via the CM web interface: Oozie Server Configuration Safety Valve for oozie-site.xml - add two properties oozie.service.ActionService.executor.ext.classes - add additional value oozie SchemaService Workflow Extension Schemas - add additional value i'd like to be able to do this via scripting of any kind.   So far i could not find such an option - neither in CM REST API (http://cloudera.github.io/cm_api/apidocs/v10/index.html) nor OOZIE REST API (http://archive.cloudera.com/cdh4/cdh/4/oozie/WebServicesAPI.html)   Thanks Doron ... View more

We have the postgres db crashes occasional. There is no core dump. Can we add the -c flag to the pg_ctl command in the /etc/init.d/cloudera-scm-server-db file? $SU -c "pg_ctl start -c -w -D $DATA_DIR -l $SERVER_LOGFILE"   can we keep this change permanent?   Thanks Doron ... View more