Support Questions
Find answers, ask questions, and share your expertise

nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

Explorer

cdh version 5.7.6, cm version 5.7.2 

 

This morning, I was installing Livy on the cluster, changed some configuration, tried to restart the whole cluster, then the YARN service could not start, tried many times, and always reported errors, but I found all the configurations, and did not find spark_shuffle.

What happened?  can anyone help me ?

 

 

---error info ------

 

Start a role 
Role failed to start due to error com.cloudera.cmf.service.config.ConfigGenException: Conflicting yarn extensions of key [spark_shuffle].
 
3 REPLIES 3

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

Expert Contributor

Hello @zbz,

 

This seems to be a known issue which was fixed on CM 5.8 and CDH 5.7.1. In your environment, please try to upgrade your CM version to 5.8.5, which was a stable version for CM 5.8.

 

Hope that helps.

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

Explorer

thanks first.

I uninstall spark1, then install spark2.1,everything is OK

Re: nodemanager can not start with error 'Conflicting yarn extensions of key [spark_shuffle].'

Expert Contributor

Hello @zbz,

 

It is nice to hear that.

 

A side node: CM version 5.7.6 is a stable release for CM 5.7. If you don't plan to upgrade to next major release then you may want to consider to upgrade to CM 5.7.6 version atleast.

 

Have a good day.