Error when distributing Parcel file

JinGon — Mon, 27 Jun 2022 02:55:52 GMT

When try to distribute Parcel file, I got errors like below.

<< cloudera-scm-agent/cloudera-flood.log >>
[27/Jun/2022 09:44:18 +0000] 23024 Thread-2 server INFO stats_alert CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel: [1000] 0 0 0 0 120 0 0 80 0 0
[27/Jun/2022 09:44:30 +0000] 23024 MainThread server INFO Adding torrent: file:///data/opt/cloudera/parcel-cache/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel.torrent
[27/Jun/2022 09:44:30 +0000] 23024 MainThread rpc ERROR Failed handling RPC: AddTorrent Already present torrent: CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
Traceback (most recent call last):
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/flood/util/rpc.py", line 176, in executeRPC
return fn(*args)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/flood/server.py", line 278, in addTorrent
torrent = self.depot.importTorrent(torrentUrl)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/flood/depot.py", line 101, in importTorrent
raise TorrentError("Already present torrent: %s", torrentName)
TorrentError: Already present torrent: CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel

<< cloudera-scm-agent/cloudera-scm-agent.log >>
[27/Jun/2022 09:44:45 +0900] 22914 WorkerThread downloader INFO Finished download [ url: http://cdh01:7180/cmf/parcel/download/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel, state: exception, total_bytes: 2136972894, downloaded_bytes: 2136972894, start_time: 2022-06-27 09:44:45, download_end_time: 2022-06-27 09:44:45, end_time: 2022-06-27 09:44:45, code: 601, exception_msg: Src file /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel does not exist, path: /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel ]
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader INFO Fetching torrent: http://cdh01:7180/cmf/parcel/download/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel.torrent
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader INFO Starting download of: http://cdh01:7180/cmf/parcel/download/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader INFO Failed adding torrent: file:///data/opt/cloudera/parcel-cache/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel.torrent Already present torrent: CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader INFO Current state: CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel [totalDownloaded=2136972894 totalSize=2136972894 upload=0 state=seeding seed=[] location=/data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel progress=1000000]
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader INFO Completed download of http://cdh01:7180/cmf/parcel/download/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel code=200 state=downloaded
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread parcel_cache WARNING No checksum in header, skipping verification
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread parcel_cache INFO Unpacking /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel into /data/opt/cloudera/parcels
[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread downloader ERROR Failed op: Src file /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel does not exist
Traceback (most recent call last):
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/downloader.py", line 502, in callable
callback(url, curr_op)
File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel_cache.py", line 200, in cb
raise e
Exception: Src file /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel does not exist

So, I checked the file "CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel" exist when distributing parcels.

[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 0
-rw-r--r-- 1 root root 0 Jun 27 09:41 diag.json
[user@dh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 193080
-rw-r--r-- 1 root root 2134900736 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 56658 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 347704
-rw-r--r-- 1 root root 2134900736 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 56336 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 622520
-rw-r--r-- 1 root root 2135425024 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 56555 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 708128
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 56466 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1013280
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 56227 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1141792
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 55716 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1304096
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 55727 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1563680
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 55071 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1744928
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 54797 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 1954048
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 54377 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2081828
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 54089 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2086948
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 0 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2086948
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 0 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2086948
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2086948
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 2086948
-rw-r--r-- 1 root root 2136972894 Jun 27 09:41 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 56
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 56
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$ ll
total 56
-rw-r--r-- 1 root root 53877 Jun 27 09:41 diag.json
[user@cdh03 CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel]$

The file "CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel" copied normally from cdh01 server.

But, a few seconds later the file "CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel" disappear.

I've installed CDH dozens of times with Cloudera Manger. I haven't had any issues with the other servers, but I'm only having these issues with Custer I'm trying to install right now.

Can anyone help me on this issue?

Re: Error when distributing Parcel file

paras — Tue, 28 Jun 2022 15:16:35 GMT

@JinGon

The error comes up at parcel unpacking

[27/Jun/2022 09:45:00 +0900] 22914 WorkerThread parcel_cache INFO Unpacking /data/opt/cloudera/parcels/.flood/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel/CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel into /data/opt/cloudera/parcels

1. Make sure you have enough space left on your hosts even after the parcel is downloaded; more than twice the parcel size for the unpacking to be successful.

2. To verify the same and also check in case the downloaded parcel file is corrupt, try to untar the parcel file on the host to some different location. If there are any of the above issues, this command would fail with appropriate errors.

# tar -xvf CDH-6.3.4-1.cdh6.3.4.p0.6751098-el7.parcel

clear the untarred contents once you are finished testing to save space.

Clear the parcel directory, restart the agent and monitor the progress.

Hope this helps,
Paras
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Re: Error when distributing Parcel file

JinGon — Wed, 29 Jun 2022 04:45:29 GMT

Hi Paras,

I have enough free space in my server.

So, I try to untar manually. It works.

Untar parcel file and move to /opt/cloudera/parcels.

And then, Cloudera Manager Agent is automatically recognize the files and it works no problems.

Thank you for your efforts.

Regards,

JinGon.

question Re: Error when distributing Parcel file in Support Questions

Error when distributing Parcel file

Re: Error when distributing Parcel file

Re: Error when distributing Parcel file