Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Highlighted

Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Explorer

After my sandbox reset

docker stop sandbox
docker rm sandbox

I've got error:

Diagnostics: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1875268269-172.17.0.2-1493988757398:blk_1073742717_1896 file=/hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

I've checked from Ambari UI, Ambari also raise error (500). Where I can download the file ?

4 REPLIES 4
Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Super Guru
@Triffids G

See if you can find this file in your local file system under /usr/hdp/current/spark2/spark2-client. If it is there then you can copy it from this location to hdfs using following command (remove existing corrupted file first).

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

hdfs dfs -put /usr/hdp/current/spark2/spark2-client/<file name> /hdp/apps/2.6.0.3-8/spark2

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Hi @Triffids G,

As mentioned by @mqureshi, remove the corrupted file first by running

hdfs dfs -rm hdp/apps/2.6.0.3-8/spark2/spark2-hdp-yarn-archive.tar.gz

You can build the proper tar file from the existing jars

cd /usr/hdp/2.6.0.3-8/spark2/jars

# create tar file from existing jars
tar -czvf spark2-hdp-yarn-archive.tar.gz *

# put the new tar file in hdfs
hdfs dfs -put spark2-hdp-yarn-archive.tar.gz /hdp/apps/2.6.0.3-8/spark2

Thanks,

Aditya

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

Explorer

Thanks, adding /usr/hdp/2.6.0.3-8/spark2/jars into archive solved the issue. Spark jobs is running OK now.

Highlighted

Re: Where I can download spark2-hdp-yarn-archive.tar.gz version 2.6.0.3-8 ?

@Triffids G,

Can you please accept the answer. This will be helpful for the users to directly check the answer instead of reading the whole thread.

Thanks,

Aditya

Don't have an account?
Coming from Hortonworks? Activate your account here