Support Questions
Find answers, ask questions, and share your expertise

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Highlighted

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Explorer

After my sandbox reset

docker stop sandbox
docker rm sandbox

I've got error:

Diagnostics: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1875268269-172.17.0.2-1493988757398:blk_1073742717_1896 file=/hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

I've checked from Ambari UI, Ambari also raise error (500). Where I can download the file ?

4 REPLIES 4

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Super Guru
@Triffids G

See if you can find this file in your local file system under /usr/hdp/current/spark2/spark2-client. If it is there then you can copy it from this location to hdfs using following command (remove existing corrupted file first).

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

hdfs dfs -put /usr/hdp/current/spark2/spark2-client/<file name> /hdp/apps/2.6.0.3-8/spark2

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Hi @Triffids G,

As mentioned by @mqureshi, remove the corrupted file first by running

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

You can build the proper tar file from the existing jars

cd /usr/hdp/2.6.0.3-8/spark2/jars

# create tar file from existing jars
tar -czvf spark2-hdp-yarn-archive.tar.gz *

# put the new tar file in hdfs
hdfs dfs -put spark2-hdp-yarn-archive.tar.gz /hdp/apps/2.6.0.3-8/spark2

Thanks,

Aditya

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Explorer

Thanks, adding /usr/hdp/2.6.0.3-8/spark2/jars into archive solved the issue. Spark jobs is running OK now.

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

@Triffids G,

Can you please accept the answer. This will be helpful for the users to directly check the answer instead of reading the whole thread.

Thanks,

Aditya