Member since
09-05-2016
16
Posts
1
Kudos Received
0
Solutions
05-11-2017
01:48 AM
My ip address has changed of my 3 node hadoop cluster installed using Cloudera Manager.CDH 5.11 What steps should I follow to make my hadoop cluster up and running??
... View more
Labels:
04-21-2017
03:51 AM
Thanks shalaj.You saved my life
... View more
04-20-2017
04:04 AM
Hi, i am trying to setup 3 node cluter using cloudera manager , getting below issue while installing we already verified all steps given below and also check older post related to the same issue but no luck. Installation failed. Failed to receive heartbeat from agent.
Ensure that the host's hostname is configured properly.
Ensure that port 7182 is accessible on the Cloudera Manager Server (check firewall rules).
Ensure that ports 9000 and 9001 are not in use on the host being added.
Check agent logs in /var/log/cloudera-scm-agent/ on the host being added. (Some of the logs can be found in the installation details).
If Use TLS Encryption for Agents is enabled in Cloudera Manager (Administration -> Settings -> Security), ensure that /etc/cloudera-scm-agent/config.ini has use_tls=1 on the host being added. Restart the corresponding agent and click the Retry link here. I don't why below image is not visible other it is public also.It is for same error above. Starting Cloudera Manager Agent...
BEGIN /sbin/service cloudera-scm-agent status
● cloudera-scm-agent.service - LSB: Cloudera SCM Agent
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-agent; bad; vendor preset: disabled)
Active: inactive (dead)
Docs: man:systemd-sysv-generator(8)
END (3)
BEGIN /sbin/service cloudera-scm-agent start
Starting cloudera-scm-agent (via systemctl): [ OK ]
END (0)
agent logs:
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.out | sed 's/^/>>/'
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO SCM Agent Version: 5.11.0
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent WARNING Expected mode 0751 for /run/cloudera-scm-agent but was 0755
>>[20/Apr/2017 16:13:27 +0000] 4664 MainThread agent INFO Re-using pre-existing directory: /run/cloudera-scm-agent
END (0)
BEGIN tail -n 50 /var/log/cloudera-scm-agent//cloudera-scm-agent.log | sed 's/^/>>/'
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
tail: cannot open '/var/log/cloudera-scm-agent//cloudera-scm-agent.log' for reading: No such file or directory
END (0)
end of agent logs.
scm agent started
Installation script completed successfully.
all done
closing logging file descriptor logs of cloudera-scm-server below 2017-04-20 16:10:36,402 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_ROOT (PT0S) to DETECT_DISTRO
2017-04-20 16:10:36,402 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_DISTRO (PT0S) to DETECT_SCM
2017-04-20 16:10:37,403 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from DETECT_SCM (PT1.001S) to REPO_INSTALL
2017-04-20 16:10:37,403 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from REPO_INSTALL (PT0S) to REFRESH_METADATA
2017-04-20 16:11:43,426 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from REFRESH_METADATA (PT66.023S) to PACKAGE_INSTALL jdk
2017-04-20 16:11:46,409 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from REFRESH_METADATA (PT70.021S) to PACKAGE_INSTALL jdk
2017-04-20 16:11:56,393 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from REFRESH_METADATA (PT80.027S) to PACKAGE_INSTALL jdk
2017-04-20 16:12:03,433 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL jdk (PT20.007S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:04,433 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT1S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:12:13,419 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL jdk (PT27.009S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:27,403 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL jdk (PT31.010S) to PACKAGE_INSTALL oracle-j2sdk1.7
2017-04-20 16:12:53,435 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT40.017S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:13:17,422 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL oracle-j2sdk1.7 (PT50.019S) to PACKAGE_INSTALL cloudera-manager-agent
2017-04-20 16:13:26,470 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT82.037S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT1.001S) to INSTALL_JCE
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:13:27,471 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:13:27,768 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from AGENT_START (PT0.297S) to SCRIPT_SUCCESS
2017-04-20 16:13:27,768 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Transitioning from SCRIPT_SUCCESS (PT0S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:14:28,136 INFO NodeConfiguratorThread-1-2:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_masternode: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:14:28,136 INFO NodeConfiguratorThread-1-2:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:17:17,648 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2017-04-20 16:17:17,652 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2015-04-21T10:47:17.649Z to reap.
2017-04-20 16:17:17,654 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Wizard is active, not reaping scanners or configurators
2017-04-20 16:17:24,541 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT247.119S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT0.001S) to INSTALL_JCE
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:17:24,542 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:17:25,510 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from AGENT_START (PT0.968S) to SCRIPT_SUCCESS
2017-04-20 16:17:25,511 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Transitioning from SCRIPT_SUCCESS (PT0.001S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:17:38,559 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL cloudera-manager-agent (PT285.124S) to PACKAGE_INSTALL cloudera-manager-daemons
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from PACKAGE_INSTALL cloudera-manager-daemons (PT1.001S) to INSTALL_JCE
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from INSTALL_JCE (PT0S) to AGENT_CONFIGURE
2017-04-20 16:17:39,560 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from AGENT_CONFIGURE (PT0S) to AGENT_START
2017-04-20 16:17:40,371 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from AGENT_START (PT0.811S) to SCRIPT_SUCCESS
2017-04-20 16:17:40,371 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Transitioning from SCRIPT_SUCCESS (PT0S) to WAIT_FOR_HEARTBEAT
2017-04-20 16:18:25,898 INFO NodeConfiguratorThread-1-1:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode2: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:18:25,898 INFO NodeConfiguratorThread-1-1:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:18:40,748 INFO NodeConfiguratorThread-1-0:com.cloudera.server.cmf.node.NodeConfiguratorProgress: sg_slavenode1: Setting WAIT_FOR_HEARTBEAT as failed and done state
2017-04-20 16:18:40,748 INFO NodeConfiguratorThread-1-0:net.schmizz.sshj.transport.TransportImpl: Disconnected - BY_APPLICATION
2017-04-20 16:21:07,825 INFO 915912177@scm-web-3:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from xxx.xx.xx.xx
2017-04-20 16:27:17,676 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Reaped total of 0 deleted commands
2017-04-20 16:27:17,680 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Found no commands older than 2015-04-21T10:57:17.676Z to reap.
2017-04-20 16:27:17,682 INFO StaleEntityEviction:com.cloudera.server.cmf.StaleEntityEvictionThread: Wizard is active, not reaping scanners or configurators
2017-04-20 16:27:33,334 INFO ScmActive-0:com.cloudera.server.cmf.components.ScmActive: (119 skipped) ScmActive completed successfully.
... View more
Labels:
03-21-2017
03:33 AM
Thanks bgooley for your response. I removed this package and ran into ssh issues.So,now working on different machine. Can u tell me what must be proper steps or commands to solve package conflicts in CentOS7?
... View more
03-21-2017
03:28 AM
I am using Hive2. I resolved it by creating temporary function per session of Hive query.
... View more
03-15-2017
10:33 PM
I have followed the below link to configure hive udf :- https://www.cloudera.com/documentation/enterprise/5-6-x/topics/cm_mc_hive_udf.html When I run my query with Hive Editor in Hue.It recognises the UDF but when running the same query with hive action oozie it gives the error :- 4924 [main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: SemanticException Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.
Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 26:44 Invalid function 'UDF_DateFormat'
org.apache.hadoop.hive.ql.parse.SemanticException: Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns. Also check for circular dependencies.
Underlying error: org.apache.hadoop.hive.ql.parse.SemanticException: Line 26:44 Invalid function 'UDF_DateFormat'
at org.apache.hadoop.hive.ql.parse.WindowingComponentizer.next(WindowingComponentizer.java:94)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genWindowingPlan(SemanticAnalyzer.java:12051)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8820)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8778)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9645)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9525)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9552)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9538)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10011)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10022)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9902)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:223)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:446)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:312)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1201)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1296)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1127)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1115)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:172)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:383)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:318)
at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:416)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:432)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:726)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:693)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628)
at org.apache.oozie.action.hadoop.HiveMain.runHive(HiveMain.java:325)
at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:302)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:49)
at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) and before this in stdout logs it says:- 3132 [main] INFO hive.ql.metadata.Hive - Registering function udf_dateformat com.org.tes.DateFormat
... View more
Labels:
03-08-2017
03:55 AM
Transaction check error: file /usr/lib64/libkadm5clnt_mit.so.8.0 from install of libkadm5-1.14.1-27.el7_3.x86_64 conflicts with file from package krb5-libs-1.13.2-10.el7.x86_64 Operating System:CentOS Linux release 7.3.1611 (Core) CM :5.8.3 I moved /usr/lib64/libkadm5clnt_mit.so.8.0 from lib64 but error still persists.
... View more
Labels:
02-23-2017
11:07 PM
Hi bgooley, I went through this post https://community.cloudera.com/t5/Cloudera-Manager-Installation/Cannot-complete-install-on-Node-using-cloudera-installer/td-p/23725 and ran the below command python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())" and returns the correct ip and hostname.
... View more
02-23-2017
05:29 AM
Did u find the solution? Can anyone repply on this
... View more
02-10-2017
02:30 AM
1 Kudo
org.apache.oozie.coord.CoordinatorJobException: E1004: Expression language evaluation error, Unable to evaluate :${coord:current(-2)}:
at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:1006)
at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveInitial(CoordSubmitXCommand.java:816)
at org.apache.oozie.command.coord.CoordSubmitXCommand.basicResolveAndIncludeDS(CoordSubmitXCommand.java:607)
at org.apache.oozie.command.coord.CoordSubmitXCommand.submitJob(CoordSubmitXCommand.java:230)
at org.apache.oozie.command.coord.CoordSubmitXCommand.submit(CoordSubmitXCommand.java:204)
at org.apache.oozie.command.SubmitTransitionXCommand.execute(SubmitTransitionXCommand.java:82)
at org.apache.oozie.command.SubmitTransitionXCommand.execute(SubmitTransitionXCommand.java:30)
at org.apache.oozie.command.XCommand.call(XCommand.java:286)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:332)
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:261)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.Exception: Unable to evaluate :${coord:current(-2)}:
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:743)
at org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:1002)
... 14 more
Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the name "coord:current"
at org.apache.commons.el.Logger.logError(Logger.java:481)
at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:525)
at org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:150)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:263)
at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:204)
at org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:734)
... 15 more
2017-02-10 16:00:02,543 INFO org.apache.oozie.command.bundle.BundleStatusUpdateXCommand: SERVER[mac99] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000048-170208193316093-oozie-oozi-B] ACTION[] Updated Bundle action [0000048-170208193316093-oozie-oozi-B_C_JSON_PCM_Log_RunningApplicationsData], status = [FAILED], pending = [0]
2017-02-10 16:00:02,544 ERROR org.apache.oozie.command.coord.CoordSubmitXCommand: SERVER[mac99] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000048-170208193316093-oozie-oozi-B] ACTION[] XException,
org.apache.oozie.command.CommandException: E1004: Expression language evaluation error, Unable to evaluate :${coord:current(-2)}:
... View more
Labels:
12-06-2016
09:01 PM
My table is an external table.Following is the alter table command ALTER TABLE db.table ADD PARTITION (date='07-12-2016') ; Exception :-- Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=admin, access=WRITE, inode="/user/admin/..../...":root:admin:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:281) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:262) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:242) at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:169) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:152) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6632) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6614) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:6566) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:4359) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:4329) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:4302) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:869) at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.mkdirs(AuthorizationProviderProxyClientProtocol.java:323) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:608) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080) )
... View more
Labels:
11-30-2016
04:01 AM
I am running it on cent os Following is memory information cat /proc/meminfo MemTotal: 7994408 kB MemFree: 143104 kB Buffers: 145996 kB Cached: 1352220 kB SwapCached: 36188 kB Active: 5636696 kB Inactive: 1819780 kB Active(anon): 4885676 kB Inactive(anon): 1100964 kB Active(file): 751020 kB Inactive(file): 718816 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 16383996 kB SwapFree: 15438660 kB Dirty: 1664 kB Writeback: 0 kB AnonPages: 5931276 kB Mapped: 129448 kB Shmem: 28380 kB Slab: 238896 kB SReclaimable: 190864 kB SUnreclaim: 48032 kB KernelStack: 26464 kB PageTables: 39800 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 20381200 kB Committed_AS: 12252892 kB VmallocTotal: 34359738367 kB VmallocUsed: 345336 kB VmallocChunk: 34359328604 kB HardwareCorrupted: 0 kB AnonHugePages: 5263360 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 9856 kB DirectMap2M: 2021376 kB DirectMap1G: 6291456 kB
... View more
11-30-2016
03:57 AM
Followin is my /etc/hadoop/conf/mapred-site.xml <property> <name>mapreduce.map.memory.mb</name> <value>0</value> </property> <property> <name>mapreduce.reduce.memory.mb</name> <value>0</value> </property> <property> <name>yarn.app.mapreduce.am.resource.mb</name> <value>1024</value> </property> <property> <name>mapreduce.map.java.opts</name> <value>-Djava.net.preferIPv4Stack=true</value> </property> <property> <name>mapreduce.reduce.java.opts</name> <value>-Djava.net.preferIPv4Stack=true</value> </property> <property> <name>yarn.app.mapreduce.am.command-opts</name> <value>-Djava.net.preferIPv4Stack=true -Xmx825955249</value> </property>
... View more
11-29-2016
10:20 PM
Sorry that was silly mistake.I should have understood that. yarn node -status taha99:8041 o/p:- 16/11/30 17:26:33 INFO client.RMProxy: Connecting to ResourceManager at taha99/172.*.*.*:8032 Node Report : Node-Id : taha99:8041 Rack : /default Node-State : RUNNING Node-Http-Address : taha99:8042 Last-Health-Update : Wed 30/Nov/16 05:26:01:543IST Health-Report : Containers : 0 Memory-Used : 0MB Memory-Capacity : 1024MB CPU-Used : 0 vcores CPU-Capacity : 2 vcores Node-Labels :
... View more
11-29-2016
05:10 AM
yarn node -list O/p:- 16/11/30 00:15:46 INFO client.RMProxy: Connecting to ResourceManager at taha99/172.*.*.*:8032 Total Nodes:1 Node-Id Node-State Node-Http-Address Number-of-Running-Containers taha99:8041 RUNNING taha99:8042 ================================================================= yarn node -status standalone:42578 O/P:- 16/11/30 00:17:43 INFO client.RMProxy: Connecting to ResourceManager at taha99/172.*.*.*:8032 Could not find the node report for node id : standalone:42578
... View more
11-29-2016
01:39 AM
I have installed Hadoop through Cloudera Manager on Single Node. I designed my workflow in in oozie to run Hive queries through it.But when I submit it gets stuck at the first action of workflow that is a Hive query to add a partition intoa table.Progress shown is 50% and state is running.When I go to logs following is the message: The application might not be running yet or there is no Node Manager or Container available. I am able to add partition through Hive editor...Please guide
... View more
Labels:
