Support Questions
Find answers, ask questions, and share your expertise

Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Explorer

I installed CDH on Ubuntu 18.04. But today when I try to upgrade to new version 6.3.1, it failed.

 

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
cloudera-manager-daemons
The following NEW packages will be installed:
cloudera-manager-agent cloudera-manager-daemons
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,242 MB of archives.
After this operation, 1,469 MB of additional disk space will be used.
Get:1 https://archive.cloudera.com/cm6/6.3.0/ubuntu1804/apt bionic-cm6/contrib amd64 cloudera-manager-daemons all 6.3.0~1281944.ubuntu1804 [1,233 MB]
Get:2 https://archive.cloudera.com/cm6/6.3.0/ubuntu1804/apt bionic-cm6/contrib amd64 cloudera-manager-agent amd64 6.3.0~1281944.ubuntu1804 [9,504 kB]
Fetched 1,240 MB in 4min 27s (4,644 kB/s)
Selecting previously unselected package cloudera-manager-daemons.
(Reading database ... 127799 files and directories currently installed.)
Preparing to unpack .../cloudera-manager-daemons_6.3.0~1281944.ubuntu1804_all.deb ...
Unpacking cloudera-manager-daemons (6.3.0~1281944.ubuntu1804) ...
Selecting previously unselected package cloudera-manager-agent.
Preparing to unpack .../cloudera-manager-agent_6.3.0~1281944.ubuntu1804_amd64.deb ...
Unpacking cloudera-manager-agent (6.3.0~1281944.ubuntu1804) ...
Setting up cloudera-manager-daemons (6.3.0~1281944.ubuntu1804) ...
Setting up cloudera-manager-agent (6.3.0~1281944.ubuntu1804) ...
[17/Oct/2019 20:40:29 +0000] 1960 MainThread supervisor INFO Supervisor version: 3.0, pid: 843
[17/Oct/2019 20:40:29 +0000] 1960 MainThread upgrade INFO Process: 405-hbase-MASTER RUNNING 16211
/var/lib/dpkg/info/cloudera-manager-agent.postinst: line 19: 1960 Segmentation fault /opt/cloudera/cm-agent/bin/cm pre_upgrade
dpkg: error processing package cloudera-manager-agent (--configure):
installed cloudera-manager-agent package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
cloudera-manager-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

 

 

 

I find /opt/cloudera/cm-agent/bin/cm pre_upgrade will cause Segmentation fault.

 

And here is the syslog:

 

Oct 17 20:54:37 dcloud001 kernel: [711276.686010] python2[7332]: segfault at ffffffffdff4b610 ip 00007f236c896818 sp 00007ffd28404230 error 5 in libptrace_do.so[7f236c895000+4000]

3 REPLIES 3
Highlighted

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Master Collaborator

@FengYu Could you please check the permission on the below file. As the error message below is coming when script is trying to set the permission on 

/var/lib/cloudera-scm-agent
/var/log/cloudera-scm-agent
/var/lib/dpkg/info/cloudera-manager-agent.postinst: line 19: 1960 Segmentation fault /opt/cloudera/cm-agent/bin/cm pre_upgrade

The below error more sounds like a corrupt package issue. You may need to install the fresh packages of Cloudera agent.

dpkg: error processing package cloudera-manager-agent (--configure):
installed cloudera-manager-agent package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
cloudera-manager-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

 


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Highlighted

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Explorer

Thanks. Here is the permission:

 

# ls -l -d /var/lib/cloudera-scm-agent /var/log/cloudera-scm-agent
drwxr-xr-x 2 cloudera-scm cloudera-scm 4096 Oct 22 00:50 /var/lib/cloudera-scm-agent
drwxr-x--- 2 cloudera-scm cloudera-scm 4096 Oct 22 09:33 /var/log/cloudera-scm-agent

 

And also, I try to delete all apt cache and use apt install -y cloudera-scm-agent again, still segment fault.

 

I have to modify /opt/cloudera/cm-agent/bin/cm to 

 

#!/bin/bash

echo

 

Then:

 

apt install -f

 

to force skip pre_upgrade, and restore /opt/cloudera/cm-agent/bin/cm to make sure service could be started, and everything works very well. Seems that the python from /opt/cloudera/cm-agent/bin/python has some troubles.

 

And I try to execute post installation script manually, also segment fault:

 

# /opt/cloudera/cm-agent/bin/cm pre_upgrade
[22/Oct/2019 10:53:59 +0000] 28844 MainThread supervisor   INFO     Supervisor version: 3.0, pid: 843
[22/Oct/2019 10:53:59 +0000] 28844 MainThread upgrade      INFO     Process: 466-hbase-REGIONSERVER RUNNING 29713
Segmentation fault

 

Highlighted

Re: Segment fault when upgrade cloudera-manager-agent to new version 6.3.1.

Master Collaborator

@FengYu Yes, force installation/skipping pre_upgrade is also an alternative. We will take a look of this and let you know if there is any issue with the script. 

 

Thanks.


Cheers!
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Don't have an account?