Member since
09-05-2016
16
Posts
1
Kudos Received
0
Solutions
04-21-2017
03:51 AM
Thanks shalaj.You saved my life
... View more
04-20-2017
04:04 AM
Hi, i am trying to setup 3 node cluter using cloudera manager , getting below issue while installing we already verified all steps given below and also check older post related to the same issue but no luck. Installation failed. Failed to receive heartbeat from agent.
Ensure that the host's hostname is configured properly.
Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
Ensure that ports 9000 and 9001 are not in use on the host being added.
Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details).
If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. I don't why below image is not visible other it is public also.It is for same error above. Starting Cloudera Manager Agent...
BEGIN /sbin/service cloudera-scm-agent status
● cloudera-scm-agent.service - LSB: Cloudera SCM Agent
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-agent; bad; vendor preset: disabled)
Active: inactive (dead)
Docs: man:systemd-sysv-generator(8)
END (3)
BEGIN /sbin/service cloudera-scm-agent start
Starting cloudera-scm-agent (via systemctl): [ OK ]
END (0)
agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
END (0)
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
END (0)
end of agent logs.
scm agent started
Installation script completed successfully.
all done
closing logging file descriptor logs of cloudera-scm-server below 2017-04-20 16:10:36,402 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_ROOT (PT0S) to DETECT_DISTRO
2017-04-20 16:10:36,402 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_DISTRO (PT0S) to DETECT_SCM
2017-04-20 16:10:37,403 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_SCM (PT1.001S) to REPO_INSTALL
2017-04-20 16:10:37,403 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from REPO_INSTALL (PT0S) to REFRESH_METADATA
2017-04-20 16:11:43,426 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from REFRESH_METADATA (PT66.023S) to PACKAGE_INSTALL jdk
2017-04-20 16:11:46,409 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from REFRESH_METADATA (PT70.021S) to PACKAGE_INSTALL jdk
2017-04-20 16:11:56,393 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from REFRESH_METADATA (PT80.027S) to PACKAGE_INSTALL jdk
2017-04-20 16:12:03,433 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL jdk (PT20.007S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:04,433 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT1S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:12:13,419 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL jdk (PT27.009S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:27,403 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL jdk (PT31.010S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:53,435 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT40.017S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:13:17,422 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT50.019S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:13:26,470 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT82.037S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT1.001S) to INSTALL_JCE
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:13:27,768 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from AGENT_START (PT0.297S) to SCRIPT_SUCCESS
2017-04-20 16:13:27,768 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from SCRIPT_SUCCESS (PT0S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:14:28,136 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:14:28,136 INFO NodeConfiguratorThread-1-2:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:17:17,648 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2017-04-20 16:17:17,652 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2015-04-21T10:47:17.649Z to reap.
2017-04-20 16:17:17,654 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Wizard is active, not reaping scanners or configurators
2017-04-20 16:17:24,541 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT247.119S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT0.001S) to INSTALL_JCE
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:17:25,510 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from AGENT_START (PT0.968S) to SCRIPT_SUCCESS
2017-04-20 16:17:25,511 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from SCRIPT_SUCCESS (PT0.001S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:17:38,559 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT285.124S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT1.001S) to INSTALL_JCE
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:17:40,371 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from AGENT_START (PT0.811S) to SCRIPT_SUCCESS
2017-04-20 16:17:40,371 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from SCRIPT_SUCCESS (PT0S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:18:25,898 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:18:25,898 INFO NodeConfiguratorThread-1-1:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:18:40,748 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:18:40,748 INFO NodeConfiguratorThread-1-0:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:21:07,825 INFO 915912177@scm-web-3:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from xxx.xx.xx.xx
2017-04-20 16:27:17,676 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2017-04-20 16:27:17,680 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2015-04-21T10:57:17.676Z to reap.
2017-04-20 16:27:17,682 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Wizard is active, not reaping scanners or configurators
2017-04-20 16:27:33,334 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: (119 skipped) ScmActive completed successfully.
... View more
Labels:
- Labels:
-
Cloudera Manager
03-21-2017
03:33 AM
Thanks bgooley for your response. I removed this package and ran into ssh issues.So,now working on different machine. Can u tell me what must be proper steps or commands to solve package conflicts in CentOS7?
... View more
03-21-2017
03:28 AM
I am using Hive2. I resolved it by creating temporary function per session of Hive query.
... View more
03-15-2017
10:33 PM
I have followed the below link to configure hive udf :- https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cm_mc_hive_udf.html When I run my query with Hive Editor in Hue.It recognises the UDF but when running the same query with hive action oozie it gives the error :- 4924 [main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: SemanticException Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.
Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 26:44 Invalid function 'UDF_DateFormat'
org.apache.hadoop.hive.ql.parse.SemanticException: Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.
Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 26:44 Invalid function 'UDF_DateFormat'
at org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:94)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:12051)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8820)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8778)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9645)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9525)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9552)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9538)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10011)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10022)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9902)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:223)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:312)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1201)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1296)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1127)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1115)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:172)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:383)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:318)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:416)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:432)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:726)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:693)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:325)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:302)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) and before this in stdout logs it says:- 3132 [main] INFO hive.ql.metadata.Hive - Registering function udf_dateformat com.org.tes.DateFormat
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Oozie
03-08-2017
03:55 AM
Transaction check error: file /usr/lib64/libkadm5clnt_mit.so.8.0 from install of libkadm5-1.14.1-27.el7_3.x86_64 conflicts with file from package krb5-libs-1.13.2-10.el7.x86_64 Operating System:CentOS Linux release 7.3.1611 (Core) CM :5.8.3 I moved /usr/lib64/libkadm5clnt_mit.so.8.0 from lib64 but error still persists.
... View more
Labels:
- Labels:
-
Cloudera Manager
