Support Questions

BrianChan · ‎05-02-2023

Hi all,

I have a cluster in CDP 7.1.8 private cloud based. Today I tried migrating CM server and following the instruction in https://docs.cloudera.com/cdp-private-cloud-base/7.1.8/managing-clusters/topics/cm-moving-the-cm-ser...

After I manually restore the cluster following the instruction in https://docs.cloudera.com/cdp-private-cloud-base/7.1.3/configuring-clusters/topics/cm-api-import-con... the cluster cannot restart.

Found that in the Parcel page in CM Console, the parcels are activating but keep on loading forever.

Here are the log message related to parcel from cloudera-scm-agent.log:

[02/May/2023 03:29:55 -0400] 6258 MainThread parcel       ERROR    Error while attempting to modify permissions of file '/opt/                                                                                cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/lib/hadoop-0.20-mapreduce/sbin/Linux/task-controller'.
  File "/opt/cloudera/cm-agent/lib/python2.7/site-packages/cmf/parcel.py", line 586, in ensure_permissions
    file = cmf.util.validate_and_open_fd(path, self.get_parcel_home(parcel))
OSError: [Errno 2] No such file or directory: '/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/lib/hadoop-0.20-mapreduc                                                                                e/sbin/Linux/task-controller'
[02/May/2023 03:29:55 -0400] 6258 MainThread downloader   INFO     Downloader path: /opt/cloudera/parcel-cache
[02/May/2023 03:29:55 -0400] 6258 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache
[02/May/2023 03:29:55 -0400] 6258 MainThread throttling_logger WARNING  Failed parsing alternatives line: sqoop-export string                                                                                 index out of range  link currently points to /opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/bin/sqoop-export
[02/May/2023 03:29:59 -0400] 6258 MainThread parcel_cache INFO     Deleting unmanaged parcel CDH-7.1.8-1.cdh7.1.8.p0.30990532
[02/May/2023 03:30:40 -0400] 6258 MainThread parcel_cache INFO     Deleting unmanaged parcel SPARK3-3.3.0.3.3.7180.0-274-1.p0.                                                                                31212967
[02/May/2023 03:30:40 -0400] 6258 MainThread parcel       INFO     prepare_environment begin: {}, [], []
[02/May/2023 03:30:40 -0400] 6258 MainThread parcel       INFO     No parcels activated for use
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:31:30 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:41:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:44:42 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:44:43 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:44:43 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 03:56:51 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 04:44:58 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 04:57:36 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {}, [], []
[02/May/2023 04:57:36 -0400] 6258 __run_queue parcel       INFO     No parcels activated for use
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:05:20 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:05:21 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:05:21 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     prepare_environment begin: {u'SPARK3': u'3.3.0.3.3.7180.0-                                                                                274-1.p0.31212967', u'CFM': u'2.1.5.0-215', u'CDH': u'7.1.8-1.cdh7.1.8.p0.30990532'}, [], []
[02/May/2023 05:13:11 -0400] 6258 __run_queue parcel       INFO     Service does not request any parcels
[02/May/2023 05:30:15 -0400] 28744 MainThread agent        INFO     To override these variables, use /etc/cloudera-scm-agent/c                                                                                onfig.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[02/May/2023 05:30:19 -0400] 28744 MainThread agent        INFO     Previously active parcels: {}
[02/May/2023 05:30:19 -0400] 28744 MainThread agent        INFO     Using parcels directory from server provided value: /opt/c                                                                                loudera/parcels
[02/May/2023 05:30:19 -0400] 28744 MainThread parcel       INFO     Agent does create users/groups
[02/May/2023 05:30:19 -0400] 28744 MainThread parcel       INFO     Agent does parcel permissions
[02/May/2023 05:30:19 -0400] 28744 MainThread downloader   INFO     Downloader path: /opt/cloudera/parcel-cache
[02/May/2023 05:30:19 -0400] 28744 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache
[02/May/2023 05:30:19 -0400] 28744 MainThread throttling_logger WARNING  Failed parsing alternatives line: sqoop-export string                                                                                 index out of range  link currently points to /opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/bin/sqoop-export
[02/May/2023 05:31:59 -0400] 29223 MainThread agent        INFO     To override these variables, use /etc/cloudera-scm-agent/c                                                                                onfig.ini. Environment variables for CDH locations are not used when CDH is installed from parcels.
[02/May/2023 05:32:03 -0400] 29223 MainThread agent        INFO     Previously active parcels: {}
[02/May/2023 05:32:03 -0400] 29223 MainThread agent        INFO     Using parcels directory from server provided value: /opt/c                                                                                loudera/parcels

Please let me know if I need to provide any further information. Thank you.

BrianChan · ‎05-09-2023

Thank you for your help @kolli_sandeep . Finally I made an API call to interrupt the Parcels activation, then remove the corresponding Parcels file and the ./flood folder in /opt/cloudera/parcels.

The parcels can successfully redistribute and activate after that.

View solution in original post

Brian, · ‎05-02-2023

Hi Brian,

Hope the /opt file system permissions are correct. Also make sure you got enough space.

[Errno 2] No such file or directory: '/opt/cloudera/parcels/CDH-7.1.8-1.cdh7.1.8.p0.30990532/

BrianChan · ‎05-09-2023

Thank you for your help @kolli_sandeep . Finally I made an API call to interrupt the Parcels activation, then remove the corresponding Parcels file and the ./flood folder in /opt/cloudera/parcels.

The parcels can successfully redistribute and activate after that.

wjsandman · ‎02-27-2024

Could you post the API call you used please? I'm having the. same issue and see nothing in the API documenttion that would allow interrupting the Parcel Activation.

DianaTorres · ‎02-27-2024

@wjsandman As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post. Thanks.

Regards,

Diana Torres,
Community Moderator

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Community Guidelines
How to use the forum

wjsandman · ‎02-29-2024

I had the exact issue described here.

What ended up working for me was

curl -k -u userid:password -X POST https://yourhost-yourdomain.com:7183/api/v50/clusters/sandboxCluster/parcels/products/CFM/versions/2...

This, obviously, was for Cloudera Flow Manafer 2.1.6.0-323

wjsandman · ‎02-29-2024

Maybe so, but a huge flaw of shared communities like this is the proliferation of answers to the same questions that are variations on a theme. Brian Chan had the exact issue I was seeing. Putting the answer to the problem WITH the stated problem makes more sense that simply parroting his question. Linking the two post is indeed better, but the one is forced to hop back and forth between posts for context.

Support Questions

Parcel stuck in activation state