Support Questions

Find answers, ask questions, and share your expertise

com.esotericsoftware.kryo.KryoException: Buffer underflow in Spark 2.x

avatar
Explorer

while running the spark job , we are facing "com.esotericsoftware.kryo.KryoException: Buffer underflow."  .

this is happening intermittently . 

 

Logic :Read few tables via spark-sql and write into hdfs using  DS.insertinto(..).  here insertInto only makes an action in the flow of the code.

 

Analysis : As checked in spark-history UI ,while execution moves from one stage to another stage it happens.

 

Env: CDH 7.x., Spark 2.x with Java

0 REPLIES 0