Reply
Highlighted
Explorer
Posts: 7
Registered: ‎04-23-2018

ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

[ Edited ]

We are trying to upgrade CDH 5.x running on Ubuntu 14.04 to Ubuntu 16.04.

 

I posted another thread about this here.

 

However, after the upgrade the cloudera-scm-agent service doesn't start. The error in supervisord.out is:

Traceback (most recent call last):
File "/usr/lib/cmf/agent/build/env/bin/supervisord", line 8, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/cmf/agent/build/env/lib/python2.7/site-packages/pkg_resources/__init__.py", line 36, in <module>
import plistlib
File "/usr/lib/python2.7/plistlib.py", line 62, in <module>
import datetime
ImportError: No module named datetime

 

I have tried with this CDH 5.10 as well as CDH 5.14

 

Please help.

Explorer
Posts: 15
Registered: ‎10-12-2015

Re: ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

I'm thinking some clues are to be had here:

https://askubuntu.com/questions/509283/python-no-module-named-datetime

 

Notice how they point out this happens after 14.10, which is of course what we did by going from 14.04 to 16.04

 

I haven't yet been able to apply anything here (since I'm trying to figure out which virtualenv's are being used or run...)

Explorer
Posts: 7
Registered: ‎04-23-2018

Re: ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

ok, the following seems to be working:

After you get the cloudera-scm-server working (after upgrade) as mentioned in the previous thread, login to cloudera manager UI, go to hosts tab and re-run the upgrade wizard. This will re-install cloudera-scm-agent on all servers. This almost worked except that on the master, the cloudera-scm-agent installation failed. I have manually copied the agent directory from one of the servers and that seems to have resolved the problem.

Explorer
Posts: 15
Registered: ‎10-12-2015

Re: ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

[ Edited ]

Thanks!  I was butting my head earlier trying to update the python env's but there's just too many of them. Thought I'd tried & failed on the wizard upgrade earlier, but I must not have, because when I tried anyway it worked fine apart from the CM node.

 

OK, I got the same thing, actually with the following error message for that first node:

 

Selection_0063.png

 

Can you be more specific on which directory is the agent directory that you manually copied from one of the successfully updated nodes?

 

ALTERNATIVELY:  Based on the details of the error message I got, which said that _apt couldn't see the trusted key, I went through and altered the ownership of the file in question:

 

root@node1:/etc/apt/trusted.gpg.d# ls -l
total 4
-rw------- 1 root root 1155 May 24 2016 cloudera-cm5.gpg
root@node1:/etc/apt/trusted.gpg.d# grep apt /etc/passwd
_apt:x:113:65534::/nonexistent:/bin/false
root@node1:/etc/apt/trusted.gpg.d# chown _apt:root cloudera-cm5.gpg
root@node1:/etc/apt/trusted.gpg.d# ls -l
total 4
-rw------- 1 _apt root 1155 May 24 2016 cloudera-cm5.gpg
root@node1:/etc/apt/trusted.gpg.d#

 

And I retried the package install on that node and it worked then.

 

 

 

 

Explorer
Posts: 15
Registered: ‎10-12-2015

Re: ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

Are you able to restart your cluster? At this point, no matter what I try to start (cluster is fully stopped, all hosts are deemed healthy), I get something like this

 

Selection_0070.png

Explorer
Posts: 15
Registered: ‎10-12-2015

Re: ImportError: No module named datetime in supervisord.out after upgrade to Ubuntu 16.04

[ Edited ]

I fixed my failure to start roles (after getting through the datetime problem) by going into parcels, downloading the 5.14 parcels, distributing them, and then activating them. Now, why THIS IS NOT PART OF THE UPGRADE WIZARD PROCESS, I have no idea.  But my cluster is up and functional, from an Ubuntu do-release-upgrade on 14.04 to 16.04 at long last.  (I replied here instead of a new thread because this was really a long set of steps on one issue: upgrading an existing cluster running on Ubuntu 14.04 to Ubuntu 16.04.)

 

 

I really hope this set of posts helps other folks, and I'd like to take a moment to send a big THANK YOU to ps40 who pointed the way when I got stuck.

 

Good luck to the rest of you. Plan on downtime and lots of troubleshooting.

Announcements