Skip to main content
h2rr821
h2rr821
Silver ⭐️⭐️
July 26, 2023
Solved

in-place upgrade CM problem(CM 7.4.4 to CM 7.7.1)

Redhat 7 OS.

I try to do a in-place upgrade Cloudera Manager from 7.4.4 to 7.7.1 following the docs:

 

https://docs.cloudera.com/cdp-private-cloud-upgrade/latest/upgrade-cdp/topics/ug_cm_upgrade_after-cdp.html

 

I do anything, and after the upgrade, I found the cloudera-scm-agent is not stable.

 

Capture.PNG

the cloudera-scm-agent log is:

 

[26/Jul/2023 15:11:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:12:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:13:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:14:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:15:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:16:31 +0000] 1956 MainThread filesystem_map ERROR NFS server binding info failed with error :- rpcinfo: RPC: Port mapper failure - Timed out
program 100003 is not available

[26/Jul/2023 15:16:32 +0000] 1956 MainThread agent ERROR Heartbeating to mysiteaddress.com:7182 failed.

 

Any suggestions? Thank you!!

venkatsambath
venkatsambathAnswer
Gold ⭐️⭐️⭐️
July 28, 2023

Glad you resolved the issue. Just to add more context, The error indicates agent was unable to monitor the filesystem, likely the nfs is stuck. This results in agent to be stuck in monitoring the filesystem and unable to heartbeat to CM server. Hence Removing the NFS from monitored filesystem helped to resolve the issue. 

View in thread

3 replies

DianaTorres
Community Manager
DianaTorres
Community Manager
July 26, 2023

@h2rr821 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDP experts @venkatsambath and @zzeng  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
h2rr821
h2rr821Author
Silver ⭐️⭐️
July 27, 2023

After I change to monitored_nodev_filesystem_types=tmpfs in /etc/cloudera-scm-agent/config.ini,

it solve my problem. 

Translate
    venkatsambath
    Gold ⭐️⭐️⭐️
    venkatsambathAnswer
    Gold ⭐️⭐️⭐️
    July 28, 2023

    Glad you resolved the issue. Just to add more context, The error indicates agent was unable to monitor the filesystem, likely the nfs is stuck. This results in agent to be stuck in monitoring the filesystem and unable to heartbeat to CM server. Hence Removing the NFS from monitored filesystem helped to resolve the issue. 

    Translate
    Terms & ConditionsAccessibility statement