2.3.0.cloudera2-1.cdh5.13.3.p0.316101 install on 5.14.2-1.cdh5.14.2.p0.3 fails

[#1]

Hi everyone

I just upgraded by 3 node research cluster to 5.14.2-1.cdh5.14.2.p0.3. The HDFS, YARN, ZOOKEEPER, HIVE, IMPALA services got successfully installed and upgraded.

I followed the steps in https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html to install  2.3.0.cloudera2-1.cdh5.13.3.p0.316101 install on the cluster

Get the following errors. I am not sure where do I go to see any detail error logs. Nothing is shown in CM

Deploying Client Configuration

Failed to deploy client configuration to the cluster.

Execute DeployClusterClientConfig for {hdfs,yarn,spark_on_yarn,hive,spark2_on_yarn} in parallel.

Completed only 4/5 steps. First failure: Failed to execute command Deploy Client Configuration on service Spark 2

Execute command Deploy Client Configuration on service Spark 2

Deploy Client Configuration failed.

Generate and deploy client configuration.

Completed only 0/3 steps. First failure: Client configuration (id=12) on host hp8300one (id=1) exited with 1 and expected 0.

[#2]

sudo grep ERROR /var/log/cloudera-scm-agent/cloudera-scm-agent.log

[10/Jun/2018 21:46:46 +0000] 32685 MainThread util         ERROR    Lineage file not found, skipping Spark plugin creation: /etc/spark2/conf.cloudera.spark2_on_yarn/navigator.lineage.client.properties

[#3 · Accepted solution]

I resolved this by installing an earlier version of Spark2

1. Deleted Spark2 Service 

2. Deactivated, and removed distributed parcel 2.3.0.cloudera2-1.cdh5.13.3.p0.316101

3. Got earlier parcel 2.2.0.cloudera2-1.cdh5.12.0.p0.232957

4. Downloaded and distributed and activated this parcel

5. Got the CSD for this and put into /opt/cloudera/csd 

6. Installed Spark2 from CM. 

Spark2 up and running on my cluster !