Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Highlighted

Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Explorer

I installed CDH on Ubuntu 18.04. But today when I try to upgrade to new version 6.3.1, it failed.

 

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
cloudera-manager-daemons
The following NEW packages will be installed:
cloudera-manager-agent cloudera-manager-daemons
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,242 MB of archives.
After this operation, 1,469 MB of additional disk space will be used.
Get:1 https://archive.cloudera.com/cm6/6.3.0/ubuntu1804/apt bionic-cm6/contrib amd64 cloudera-manager-daemons all 6.3.0~1281944.ubuntu1804 [1,233 MB]
Get:2 https://archive.cloudera.com/cm6/6.3.0/ubuntu1804/apt bionic-cm6/contrib amd64 cloudera-manager-agent amd64 6.3.0~1281944.ubuntu1804 [9,504 kB]
Fetched 1,240 MB in 4min 27s (4,644 kB/s)
Selecting previously unselected package cloudera-manager-daemons.
(Reading database ... 127799 files and directories currently installed.)
Preparing to unpack .../cloudera-manager-daemons_6.3.0~1281944.ubuntu1804_all.deb ...
Unpacking cloudera-manager-daemons (6.3.0~1281944.ubuntu1804) ...
Selecting previously unselected package cloudera-manager-agent.
Preparing to unpack .../cloudera-manager-agent_6.3.0~1281944.ubuntu1804_amd64.deb ...
Unpacking cloudera-manager-agent (6.3.0~1281944.ubuntu1804) ...
Setting up cloudera-manager-daemons (6.3.0~1281944.ubuntu1804) ...
Setting up cloudera-manager-agent (6.3.0~1281944.ubuntu1804) ...
[17/Oct/2019 20:40:29 +0000] 1960 MainThread supervisor INFO Supervisor version: 3.0, pid: 843
[17/Oct/2019 20:40:29 +0000] 1960 MainThread upgrade INFO Process: 405-hbase-MASTER RUNNING 16211
/var/lib/dpkg/info/cloudera-manager-agent.postinst: line 19: 1960 Segmentation fault /opt/cloudera/cm-agent/bin/cm pre_upgrade
dpkg: error processing package cloudera-manager-agent (--configure):
installed cloudera-manager-agent package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
cloudera-manager-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

 

 

 

I find /opt/cloudera/cm-agent/bin/cm pre_upgrade will cause Segmentation fault.

 

And here is the syslog:

 

Oct 17 20:54:37 dcloud001 kernel: [711276.686010] python2[7332]: segfault at ffffffffdff4b610 ip 00007f236c896818 sp 00007ffd28404230 error 5 in libptrace_do.so[7f236c895000+4000]

3 REPLIES 3

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Expert Contributor

@FengYu Could you please check the permission on the below file. As the error message below is coming when script is trying to set the permission on 

/var/lib/cloudera-scm-agent
/var/log/cloudera-scm-agent
/var/lib/dpkg/info/cloudera-manager-agent.postinst: line 19: 1960 Segmentation fault /opt/cloudera/cm-agent/bin/cm pre_upgrade

The below error more sounds like a corrupt package issue. You may need to install the fresh packages of Cloudera agent.

dpkg: error processing package cloudera-manager-agent (--configure):
installed cloudera-manager-agent package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
cloudera-manager-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

 

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Explorer

Thanks. Here is the permission:

 

# ls -l -d /var/lib/cloudera-scm-agent /var/log/cloudera-scm-agent
drwxr-xr-x 2 cloudera-scm cloudera-scm 4096 Oct 22 00:50 /var/lib/cloudera-scm-agent
drwxr-x--- 2 cloudera-scm cloudera-scm 4096 Oct 22 09:33 /var/log/cloudera-scm-agent

 

And also, I try to delete all apt cache and use apt install -y cloudera-scm-agent again, still segment fault.

 

I have to modify /opt/cloudera/cm-agent/bin/cm to 

 

#!/bin/bash

echo

 

Then:

 

apt install -f

 

to force skip pre_upgrade, and restore /opt/cloudera/cm-agent/bin/cm to make sure service could be started, and everything works very well. Seems that the python from /opt/cloudera/cm-agent/bin/python has some troubles.

 

And I try to execute post installation script manually, also segment fault:

 

# /opt/cloudera/cm-agent/bin/cm pre_upgrade
[22/Oct/2019 10:53:59 +0000] 28844 MainThread supervisor   INFO     Supervisor version: 3.0, pid: 843
[22/Oct/2019 10:53:59 +0000] 28844 MainThread upgrade      INFO     Process: 466-hbase-REGIONSERVER RUNNING 29713
Segmentation fault

 

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Expert Contributor

@FengYu Yes, force installation/skipping pre_upgrade is also an alternative. We will take a look of this and let you know if there is any issue with the script. 

 

Thanks.

Don't have an account?
Coming from Hortonworks? Activate your account here