Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

New Contributor

After my sandbox reset

docker stop sandbox
docker rm sandbox

I've got error:

Diagnostics: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1875268269-172.17.0.2-1493988757398:blk_1073742717_1896 file=/hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

I've checked from Ambari UI, Ambari also raise error (500). Where I can download the file ?

4 REPLIES 4

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Super Guru
@Triffids G

See if you can find this file in your local file system under /usr/hdp/current/spark2/spark2-client. If it is there then you can copy it from this location to hdfs using following command (remove existing corrupted file first).

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

hdfs dfs -put /usr/hdp/current/spark2/spark2-client/<file name> /hdp/apps/2.6.0.3-8/spark2

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Hi @Triffids G,

As mentioned by @mqureshi, remove the corrupted file first by running

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

You can build the proper tar file from the existing jars

cd /usr/hdp/2.6.0.3-8/spark2/jars

# create tar file from existing jars
tar -czvf spark2-hdp-yarn-archive.tar.gz *

# put the new tar file in hdfs
hdfs dfs -put spark2-hdp-yarn-archive.tar.gz /hdp/apps/2.6.0.3-8/spark2

Thanks,

Aditya

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

New Contributor

Thanks, adding /usr/hdp/2.6.0.3-8/spark2/jars into archive solved the issue. Spark jobs is running OK now.

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

@Triffids G,

Can you please accept the answer. This will be helpful for the users to directly check the answer instead of reading the whole thread.

Thanks,

Aditya