About SalmanM

SalmanM · ‎02-28-2025

HI Guys, I am having an issue running an Spark 3 jobs: Driver log: ERROR: cluster:YarnClusterScheduler: [dispatcher-CoarseGrainedScheduler] : Lost executor 1 on host_name: Unable to create executor due to unable to register with external shuffle service due to: java.lang.IllegalException: Unknown message type: -22 Executor log: ERROR: Exception while bootstrapping client in 3 ms. Java.lang.RuntimeException: java.lang.IllegalException: Unknown message type: -22 Executor making 3 attempts with 5 second waiting period, then exiting. From my research, I found recommendation on increasing the shuffle register timeout. Any recommendation is appreciated.

SalmanM · ‎11-09-2023

If a have 2 original block and 1 Parity block in HDFS according to the EC policy, then If I lose 2 of this 3 blocks, How HDFS is going to Recover the missing data. Block A xor Block B = Parity Block If I lose any one of the block, I can see HDFS will be able to create the missing block. But if I lose any 2 block, I don't see how we will recover the missing 2 block out of 1 block. If help me understand or redirect me to some helpful resources.

Online	Offline
Last Visited	‎07-10-2025 06:31 PM

Member Since	‎10-16-2023 04:54 PM
Last Visited	‎07-10-2025 06:31 PM
Posts	2

Cloudera Community

Spark3 Issue: Executor is not able to register wit...

Data recovery in Erasure coding.