EOF Exception while deployed spark application on YARN.

[#1]

java.io.EOFException at java.io.DataInputStream.readShort(DataInputStream.java:315) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:58) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:227) at java.lang.Thread.run(Thread.java:748)

Versions:-

HDP 2.3

Hadoop- 2.7.1

Spark- 2.2.0

Please revert me as what kind of issue we're facing on server side and what kind of changes we need to perform?

[#2]

Hi @Rohit Khose,

This occurs usually when spark code is not formatted(either the braces or quotes/ indentation (in case of python) are not ended) properly will result this exception.

could you palest reformat your code and run the same (best is to use a text editor which highlight or submit the code block by block)

On the other hand this may occur due to the network failures while on data shuffle between execution (I doubt this is not the case) - in such cases resubmitting the job should result the successful completion(presume not in your case).