Please follow below posts and see if that resolve this issue. https://community.hortonworks.com/questions/15506/error-cannot-retrieve-repository-metadata-repomdxm.html https://community.hortonworks.com/questions/31523/facing-error-while-upgrading-from-hdp-23-to-24-usi.html ... View more

You added it in wrong way, you need to append those confs in AMBARI_JVM_ARGS variable. After that restart the ambari server. export AMBARI_JVM_ARGS=$AMBARI_JVM_ARGS' -Xms512m -Xmx2048m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Dhttp.proxyHost=proxy.sgp.hp.com -Dhttp.proxyPort=8080' ... View more

please follow this doc for ambari internet proxy setup. http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_reference_guide/content/_how_to_set_up_an_internet_proxy_server_for_ambari.html ... View more

can you please share the output of below command from that node where its failing? yum list 'hadoop_2_4*-libhdfs*' ... View more

Yes, you can use hdfs checksum to find the file integrity on hdfs. See below jira's for more information. https://issues.apache.org/jira/browse/HADOOP-9209 https://issues.apache.org/jira/browse/HDFS-219 https://issues.apache.org/jira/browse/HADOOP-3981 ... View more

@Raja Ray As per the logs I couldn't find where datanode is getting down, can you please provide more logs? Also the error might not be relevant since it seems to be related to https://issues.apache.org/jira/browse/AMBARI-12420. Check if you are using same ambari version. ... View more

Try adding spark assembly jar while running pyspark. pyspark --jars /usr/hdp/current/spark-client/lib/spark-assembly-<version>-hadoop2<version>.jar ... View more