I have upgraded from 5.16 to 6.2.0 CDH version , all is good but in the end when i try to Finalize the HDFS Upgrade ,it is failing.Can't open /var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade/supervisor_status: Permission denied.

[#1]

I have upgraded from 5.16 to 6.2.0 CDH version , all is good but in the end when i try to Finalize the HDFS Upgrade ,it is failing.

+ set -x
+ replace_conf_dir
+ echo CONF_DIR=/var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade
+ echo CMF_CONF_DIR=
+ EXCLUDE_CMF_FILES=('cloudera-config.sh' 'hue.sh' 'impala.sh' 'sqoop.sh' 'supervisor.conf' 'config.zip' 'proc.json' '*.log' '*.keytab' '*jceks')
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '*.log' hdfs.keytab '*jceks'
+ find /var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade -type f '!' -path '/var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade/logs/*' '!' -name cloudera-config.sh '!' -name hue.sh '!' -name impala.sh '!' -name sqoop.sh '!' -name supervisor.conf '!' -name config.zip '!' -name proc.json '!' -name '*.log' '!' -name hdfs.keytab '!' -name '*jceks' -exec perl -pi -e 's#\{\{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade#g' '{}' ';'
Can't open /var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade/supervisor_status: Permission denied.
+ make_scripts_executable
+ find /var/run/cloudera-scm-agent/process/6934-hdfs-NAMENODE-finalize-upgrade -regex '.*\.\(py\|sh\)$' -exec chmod u+x '{}' ';'
+ '[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']'
+ ulimit -l
+ export HADOOP_IDENT_STRING=hdfs
+ HADOOP_IDENT_STRING=hdfs
+ '[' -n '' ']'
+ '[' mkdir '!=' finalize-upgrade ']'
+ acquire_kerberos_tgt hdfs.keytab
+ '[' -z hdfs.keytab ']'

 Any insights, my cloudera manager version is 6.2.0, java version is 

java-8-oracle-cloudera

[#2]

@prabhat The error seems misleading as per my experience but I would like to check the ownership and permission on the directory  /var/run/cloudera-scm-agent/process/ this should be like below: 

ll /var/run/cloudera-scm-agent/process/ | grep -i hdfs
drwxr-x--x. 3 hdfs hdfs 420 Jun 26 15:18 1805-hdfs-DATANODE
drwxr-x--x. 4 httpfs httpfs 360 Jun 26 15:18 1806-hdfs-HTTPFS
drwxr-x--x. 3 hdfs hdfs 420 Jun 26 15:30 1857-hdfs-DATANODE
drwxr-x--x. 4 httpfs httpfs 360 Jun 26 15:30 1858-hdfs-HTTPFS
drwxr-x--x. 3 hdfs hdfs 420 Jun 26 16:40 1902-hdfs-DATANODE
drwxr-x--x. 4 httpfs httpfs 360 Jun 26 16:40 1903-hdfs-HTTPFS

If the you there is permission issue then fix them else you can try to hard start agent once and see how this goes.