Support Questions
Find answers, ask questions, and share your expertise

why the mapReduce status always is "Accepted"?

Explorer

I'm try to check if the cluster service working, then I execute the example of wordCount program which hadoop provide.

No normal exit after executing the instruction of wordcount.

image.png

 

I see the wordCount task details from UI,the "State" all of "Accepted".

image.png

 

#NodeManager log:

 

 

2019-08-30 02:59:15,027 INFO  zookeeper.ZooKeeper (ZooKeeper.java:close(684)) - Session: 0x26ce08e40590012 closed
2019-08-30 02:59:15,027 INFO  zookeeper.ClientCnxn (ClientCnxn.java:run(524)) - EventThread shut down
2019-08-30 03:00:15,027 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:01:15,028 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:02:15,029 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:03:15,030 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:04:15,030 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
c2019-08-30 03:05:15,031 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:06:15,031 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:07:15,032 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:08:01,454 INFO  localizer.ResourceLocalizationService (ResourceLocalizationService.java:handleCacheCleanup(542)) - Cache Size Before Clean: 0, Total Deleted: 0, Public Deleted: 0, Private Deleted: 0
2019-08-30 03:08:07,095 INFO  tracker.NMLogAggregationStatusTracker (NMLogAggregationStatusTracker.java:rollLogAggregationStatus(206)) - Rolling over the cached log aggregation status.
2019-08-30 03:08:15,033 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.
2019-08-30 03:09:15,034 INFO  zookeeper.ReadOnlyZKClient (ReadOnlyZKClient.java:run(315)) - 0x55ecbafe no activities for 60000 ms, close active connection. Will reconnect next time when there are new requests.

 

 

 

After I read the yarn-nodeManager log, I thought the restart was resolved, and a new exception appeared: Connection refused: hadoop7.gsta.cn/10.1.1.7:17020

 

 

Fri Aug 30 03:39:43 EDT 2019, RpcRetryingCaller{globalStartTime=1567150775111, pause=100, maxAttempts=8}, java.net.ConnectException: Call to hadoop7.gsta.cn/10.1.1.7:17020 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: hadoop7.gsta.cn/10.1.1.7:17020

        at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:145)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:80)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Call to hadoop7.gsta.cn/10.1.1.7:17020 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: hadoop7.gsta.cn/10.1.1.7:17020
        at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:165)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
        at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
        at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
        at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:92)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:329)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:315)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:307)
        at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1377)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:329)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:315)
        at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:929)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:179)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$500(NettyRpcConnection.java:71)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:269)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:263)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:327)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:343)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
        ... 1 more
Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: hadoop7.gsta.cn/10.1.1.7:17020
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:323)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
        ... 7 more
Caused by: java.net.ConnectException: Connection refused
        ... 11 more
2019-08-30 03:39:47,447 INFO  client.RpcRetryingCallerImpl (RpcRetryingCallerImpl.java:callWithRetries(134)) - Call exception, tries=6, retries=6, started=4151 ms ago, cancelled=false, msg=Call to hadoop7.gsta.cn/10.1.1.7:17020 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: hadoop7.gsta.cn/10.1.1.7:17020, details=row 'prod.timelineservice.entity,yarn-ats!yarn_cluster!ats-hbase��������������!YARN_CONTAINER���z�`!container_1567134062883_0001_01_000001,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=hadoop7.gsta.cn,17020,1567136010280, seqNum=-1

 

 

 

follow the Hbase table 'hbase:meta',I see this:
image.pngit seems should connect to 10.1.1.7:16020,isn't connect 10.1.1.7:17020.

and the hbase-site.xml details tell us,it run in 16020

image.png

Please tell me what should I do? I've been plagued by this problem all day.

1 REPLY 1

Re: why the mapReduce status always is "Accepted"?

Explorer

@LeeFan   Dear LeeFan can you please tell me how did you resolved this issue?

I am having exactly the same error: Connection refused: 10.1.165.71:17020

 

Please if you resolved it, it will mean a lot to me if you can help me.

Thank you in advance,