24/08/29 10:21:15 WARN DataStreamer: Exception for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526 Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: java.io.EOFException: Unexpected EOF while trying to read response from server Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:521) Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) Aug 29 10:21:15 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1137) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: DataStreamer Exception Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: java.net.SocketTimeoutException: 495000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/172.20.0.194:40396 remote=/172.20.0.194:9866] Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:163) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:158) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:116) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at java.io.DataOutputStream.write(DataOutputStream.java:107) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DFSPacket.writeTo(DFSPacket.java:193) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.sendPacket(DataStreamer.java:857) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:762) Aug 29 10:27:30 gws-siem-dnt-3 bash[2218954]: 24/08/29 10:27:30 WARN DataStreamer: Error Recovery for BP-942923949-172.20.0.202-1722967626672:blk_1074290600_560526 in pipeline [DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK], DatanodeInfoWithStorage[172.20.0.204:9866,DS-5a287a01-fda4-4ad0-a3e6-9d24cc6d8568,DISK], DatanodeInfoWithStorage[172.20.0.196:9866,DS-90f354a6-9531-49dd-97c4-af388f697c34,DISK]]: datanode 0(DatanodeInfoWithStorage[172.20.0.194:9866,DS-d8020c9c-24d1-462a-b707-0d219ce5848a,DISK]) is bad. last 2 months we have had the issue please help me due to this our script got stuck and the lag has increased Please help me   ... View more

  The above file is the Hadoop-policy.xml in that file we have given the properties for enabling ACL for users when I am adding users like yarn mapred hdfs etc so it throws an ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Application application_1699592375401_1753 failed 2 times due to Error launching appattempt_1699592375401_1753_000002. Got exception: org.apache.hadoop.security.authorize.AuthorizationException org. apache. Hadoop.security.authorize.AuthorizationException: User appattempt_1699592375401_1753_000002 (auth: TOKEN) is not authorized for protocol interface org. apache. Hadoop.yarn.api.ContainerManagementProtocolPB: denied by configured ACL Not able to identify which user is running the container, due to which ACL blocking When put * it is working fine Please help me with this ... View more

ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Application application_1699592375401_1753 failed 2 times due to Error launching appattempt_1699592375401_1753_000002. Got exception: org.apache.hadoop.security.authorize.AuthorizationException: User appattempt_1699592375401_1753_000002 (auth: TOKEN) is not authorized for protocol interface org. apache. Hadoop.yarn.api.ContainerManagementProtocolPB: denied by configured ACL Not able to identify which user is running the container, due to which ACL blocking When put * it is working fine Please help me on this ... View more