Can't find cloudera 5.7 parcel

[#1]

Hi, guys!

I'm trying to install cloudera using CM api. Today I started seeing this problem: can't find CDH 5.7 even though the repo is specified:

In the parcels page I see a bunch of parcels, but not the CDH 5.7 one.

Also, on another Cluster I started seeing these messages:

ReadWriteDatabaseTaskCallable

Error while executing CmfEntityManager task
java.lang.NullPointerException
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191)
	at com.google.common.collect.Collections2.filter(Collections2.java:92)
	at com.cloudera.parcel.components.ParcelDownloaderImpl$RepositoryInfo.getParcelsWithValidNames(ParcelDownloaderImpl.java:673)
	at com.cloudera.parcel.components.ParcelDownloaderImpl$RepositoryInfo.getSortedParcels(ParcelDownloaderImpl.java:691)
	at com.cloudera.parcel.components.ParcelDownloaderImpl.syncRemoteRepos(ParcelDownloaderImpl.java:368)
	at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:438)
	at com.cloudera.parcel.components.ParcelDownloaderImpl$1.run(ParcelDownloaderImpl.java:433)
	at com.cloudera.cmf.persist.ReadWriteDatabaseTaskCallable.call(ReadWriteDatabaseTaskCallable.java:36)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

The first one happened at November 10, 2017 5:59 PM.

This message is happening on the new clusters as well.

Is everything ok with the repo?

[#2 · Accepted solution]

Still don't know the exact cause of it. Fixed the issue by removing Cloudera Manager's default parcel repositories and left only that which I need

[#3]

https://archive.cloudera.com/sqoop-connectors/parcels/latest has some toxic content. Removing it from the list of remote repositories solved the problem for me.

[#4]

Hi,

Are you saying that when you removed the SQOOP-CONNECTORS  ( https://archive.cloudera.com/sqoop-connectors/parcels/latest ) URL from the parcels list you were able to see your CDH upgrade choice?

I want to report this behavior if that is true.

Thank you - Tina

[#5]

Tina, for me - removing all repos but CDH 5.7 one (including sqoop connector) solved the problem.

[#6]

Thank you for your answer.   I am going to use this information and test this.

[#7]

Any news on this? I still have problems for some deploys with https://archive.cloudera.com/cdh5/parcels/5.7/

[#8]

I cannot reproduce this.   What version of CM are you using? 

[#9]

It's CM 5.13

[#10]

I cannot reproduce this issue, even on CM5.13.0.

I can confirm that you will only see the latest released version of CDH if you only have this repo, the default, in your parcel list:

https://archive.cloudera.com/cdh5/parcels/{latest_supported}/

The latest supported release would be the one that matches your CM version.   CM5.10.x would see CDH5.10.x, CM5.13.0 would see CDH5.13.x and so on.

To see a version of CDH that is lower than the one you have running you will have to point directly to that repo.

CM5.8.x and higher, to see CDH5.7.x would need to add http://archive.cloudera.com/cdh5/parcels/5.7/ to the repo list.

As far as the sqoop repo being corrupt or causing issues, I have contacted those responsible to look into it.

[#11]

Thank you for your response! You're right, to see CDH 5.7 from CM 5.13 - I need to use the http://archive.cloudera.com/cdh5/parcels/5.7/ repo, what I actually am doing. It works almost all the time except, but sometimes - it doesn't. I saw errors, which are the same to the errors related to the sqoop repo.

Thank you for the investigation! I'll create some internal mirror for the CDH 5.7 parcels to be sure that repository is available all the time.

[#12]

Thanks for the workaround which was a great help as I got this error just today with CDH 5.12.x.

After adding the Anaconda parcel: https://repo.continuum.io/pkgs/misc/parcels/ , it didnt show up in the parcel list when Checking for New Parcels. There was a null pointer exception in cloudera-scm-server.log . Had to remove both http and https sqoop parcel urls after which the Anaconda parcel showed up.

http://archive.cloudera.com/sqoop-connectors/parcels/latest/

https://archive.cloudera.com/sqoop-connectors/parcels/latest/

[#13]

I have reproduced this issue, not sure why it took so long to affect my installs.    Removing the sqoop repo https://archive.cloudera.com/sqoop-connectors/parcels/latest/  is indeed the work around.

I have notified the correct Cloudera parties to look into and fix this issue.

Tina

[#14]

This issue should be fixed now.   I have verified that the parcels show up on my cluster.

Tina

[#15]

> Are you saying that when you removed the SQOOP-CONNECTORS  ( https://archive.cloudera.com/sqoop-connectors/parcels/latest ) URL from the parcels list you were able to see your CDH upgrade choice?

When I removed the SQOOP-CONNECTORS URL from the list of remote repositories I stopped getting the NPE and became able to install new parcels.