Ambari 2.5 upgrade and HDPSearch issue

ithoughtful — Mon, 03 Jul 2017 20:38:20 GMT

I've run into a bug when upgrading the Ambari from 2.4 to 2.5. I followed all the recommended step until I got to upgrading the Ambari Server database schema using the "ambari-server upgrade" command which failed with the following error due to a known issue when the Lucidworks HDPSearch component is previously added to Ambari:

$> ambari-server upgrade -v
Using python  /usr/bin/python
Upgrading ambari-server
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Upgrade Ambari Server
INFO: Updating Ambari Server properties in ambari.properties ...
WARNING: Can not find ambari.properties.rpmsave file from previous version, skipping import of settings
INFO: Updating Ambari Server properties in ambari-env.sh ...
INFO: Can not find ambari-env.sh.rpmsave file from previous version, skipping restore of environment settings. ambari-env.sh may not include any user customization.
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: ===========================================================================================
INFO: Executing Mpack Replay Log :
INFO: {'purge': False, 'mpack_command': 'install-mpack', 'mpack_path': '/var/lib/ambari-server/resources/mpacks/cache/solr-service-mpack-5.5.2.2.5.tar.gz', 'force': False, 'verbose': False}
INFO: ===========================================================================================
INFO: Installing management pack /var/lib/ambari-server/resources/mpacks/cache/solr-service-mpack-5.5.2.2.5.tar.gz
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Download management pack to temp location /var/lib/ambari-server/data/tmp/solr-service-mpack-5.5.2.2.5.tar.gz
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Expand management pack at temp location /var/lib/ambari-server/data/tmp/solr-service
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
INFO: Stage management pack solr-ambari-mpack-5.5.2.2.5 to staging location /var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-5.5.2.2.5
INFO: Force removing previously installed management pack from /var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-5.5.2.2.5
INFO: Processing artifact SOLR-common-services of type service-definitions in /var/lib/ambari-server/resources/mpacks/solr-ambari-mpack-5.5.2.2.5/common-services
INFO: Loading properties from /etc/ambari-server/conf/ambari.properties
Traceback (most recent call last):
  File "/usr/sbin/ambari-server.py", line 941, in <module>
    mainBody()
  File "/usr/sbin/ambari-server.py", line 911, in mainBody
    main(options, args, parser)
  File "/usr/sbin/ambari-server.py", line 863, in main
    action_obj.execute()
  File "/usr/sbin/ambari-server.py", line 78, in execute
    self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", line 363, in upgrade
    replay_mpack_logs()
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 983, in replay_mpack_logs
    install_mpack(replay_options, replay_mode=True)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 896, in install_mpack
    (mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, mpack_archive_path) = _install_mpack(options, replay_mode)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 792, in _install_mpack
    process_service_definitions_artifact(artifact, artifact_source_dir, options)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 515, in process_service_definitions_artifact
    create_symlink(src_service_definitions_dir, dest_service_definitions_dir, file, options.force)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 235, in create_symlink
    create_symlink_using_path(src_path, dest_link, force)
  File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 247, in create_symlink_using_path
    sudo.symlink(src_path, dest_link)
  File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 123, in symlink
    os.symlink(source, link_name)
OSError: [Errno 17] File exists

This issue is logged but the fix is only availble in the next release! : https://issues.apache.org/jira/browse/AMBARI-21263

Trying to rectify this, I uninstalled the Lucidworks HDPSearch from the cluster but that didn't help either.

Now what should I do? How should I either resolve this upgrade issue or at least rollback to 2.4 without messing up ?

Re: Ambari 2.5 upgrade and HDPSearch issue

jsensharma — Mon, 03 Jul 2017 23:08:46 GMT

@Alireza Sadeghi

The fix mentioned in AMBARI-21263 is very simple. You only need to modify your "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py" script as mentioned in :

https://reviews.apache.org/r/60126/diff/3#index_header

So please take a backup of your "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py"file and try editing the lines as mentioned in the fix.

NOTE: While editing the script line starting with - (Minus) need to be removed and the lines starting with + (Plus) need to be added.

Re: Ambari 2.5 upgrade and HDPSearch issue

sujek — Wed, 05 Jul 2017 19:21:57 GMT

@Alireza Sadeghi

You can apply this patch using "patch" command: http://www.tutorialspoint.com/unix_commands/patch.htm

1) Install patch application

 yum install patch

2) Download patch AMBARI-21263

wget https://issues.apache.org/jira/secure/attachment/12873572/AMBARI-21263.patch

3) Backup serverUpgrade.py file

cp /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py_org

4) Apply patch

patch < AMBARI-21263.patch

can't find file to patch at input line 5
Perhaps you should have used the -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/ambari-server/src/main/python/ambari_server/serverUpgrade.py b/ambari-server/src/main/python/ambari_server/serverUpgrade.py
|index 160c91d..8263062 100644
|--- a/ambari-server/src/main/python/ambari_server/serverUpgrade.py
|+++ b/ambari-server/src/main/python/ambari_server/serverUpgrade.py
--------------------------
File to patch: /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py
patching file /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py

5) Check files differences

diff /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py /usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py_org

question Re: Ambari 2.5 upgrade and HDPSearch issue in Support Questions

Ambari 2.5 upgrade and HDPSearch issue

Re: Ambari 2.5 upgrade and HDPSearch issue

Re: Ambari 2.5 upgrade and HDPSearch issue