Reply
Highlighted
zbz
Explorer
Posts: 16
Registered: ‎10-07-2018

nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

[ Edited ]

cdh version 5.7.6, cm version 5.7.2 

 

This morning, I was installing Livy on the cluster, changed some configuration, tried to restart the whole cluster, then the YARN service could not start, tried many times, and always reported errors, but I found all the configurations, and did not find spark_shuffle.

What happened?  can anyone help me ?

 

 

---error info ------

 

Start a role 
Role failed to start due to error com.cloudera.cmf.service.config.ConfigGenException: Conflicting yarn extensions of key [spark_shuffle].
 
Expert Contributor
Posts: 92
Registered: ‎01-08-2016

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

Hello @zbz,

 

This seems to be a known issue which was fixed on CM 5.8 and CDH 5.7.1. In your environment, please try to upgrade your CM version to 5.8.5, which was a stable version for CM 5.8.

 

Hope that helps.

zbz
Explorer
Posts: 16
Registered: ‎10-07-2018

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

thanks first.

I uninstall spark1, then install spark2.1,everything is OK

Expert Contributor
Posts: 92
Registered: ‎01-08-2016

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

[ Edited ]

Hello @zbz,

 

It is nice to hear that.

 

A side node: CM version 5.7.6 is a stable release for CM 5.7. If you don't plan to upgrade to next major release then you may want to consider to upgrade to CM 5.7.6 version atleast.

 

Have a good day.