Support Questions
Find answers, ask questions, and share your expertise

Ambari-server upgrade from hdf-2.0 to hdf-3.0

Contributor

setupMpacks.py OSError: [Errno 17] File exists

12 REPLIES 12

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Super Mentor

@Dmitro Vasilenko

Can you please share the complete stack trace of the error. That way we will know which File the error is pointing out?

It might be due to some symlink that already exist. The detailed stacktrace of the error can show us which symlink it is checking. (We might need to remove/move the symlink/directory on our own to see if it fixes the issue).

.

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Contributor

ambari-server upgrade -v

Using python /usr/bin/python

Upgrading ambari-server

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Upgrade Ambari Server

INFO: Updating Ambari Server properties in ambari.properties ...

WARNING: Can not find ambari.properties.rpmsave file from previous version, skipping import of settings

INFO: Updating Ambari Server properties in ambari-env.sh ...

INFO: Can not find ambari-env.sh.rpmsave file from previous version, skipping restore of environment settings. ambari-env.sh may not include any user customization.

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: ===========================================================================================

INFO: Executing Mpack Replay Log :

INFO: {'purge': True, 'mpack_command': 'install-mpack', 'mpack_path': '/var/lib/ambari-server/resources/mpacks/cache/hdf-ambari-mpack-2.1.1.0-2.tar.gz', 'force': False, 'verbose': True}

INFO: ===========================================================================================

INFO: Installing management pack /var/lib/ambari-server/resources/mpacks/cache/hdf-ambari-mpack-2.1.1.0-2.tar.gz

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Download management pack to temp location /var/lib/ambari-server/data/tmp/hdf-ambari-mpack-2.1.1.0-2.tar.gz

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Expand management pack at temp location /var/lib/ambari-server/data/tmp/hdf-ambari-mpack-2.1.1.0-2/

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: AMBARI_SERVER_LIB is not set, using default /usr/lib/ambari-server

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: about to run command: /usr/jdk64/jdk1.8.0_77/bin/java -cp '/etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/share/java/postgresql-jdbc.jar' org.apache.ambari.server.checks.MpackInstallChecker --mpack-stacks HDF

INFO:

process_pid=10402

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Purging existing stack definitions and management packs

INFO: Purging stack location: /var/lib/ambari-server/resources/stacks

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

INFO: Stage management pack hdf-ambari-mpack-2.1.1.0-2 to staging location /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-2.1.1.0-2

INFO: Force removing previously installed management pack from /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-2.1.1.0-2

INFO: Processing artifact hdf-service-definitions of type service-definitions in /var/lib/ambari-server/resources/mpacks/hdf-ambari-mpack-2.1.1.0-2/common-services

INFO: Loading properties from /etc/ambari-server/conf/ambari.properties

Traceback (most recent call last):

File "/usr/sbin/ambari-server.py", line 941, in <module>

mainBody()

File "/usr/sbin/ambari-server.py", line 911, in mainBody

main(options, args, parser)

File "/usr/sbin/ambari-server.py", line 863, in main

action_obj.execute()

File "/usr/sbin/ambari-server.py", line 78, in execute

self.fn(*self.args, **self.kwargs)

File "/usr/lib/python2.6/site-packages/ambari_server/serverUpgrade.py", line 363, in upgrade

replay_mpack_logs()

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 983, in replay_mpack_logs

install_mpack(replay_options, replay_mode=True)

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 896, in install_mpack

(mpack_metadata, mpack_name, mpack_version, mpack_staging_dir, mpack_archive_path) = _install_mpack(options, replay_mode)

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 792, in _install_mpack

process_service_definitions_artifact(artifact, artifact_source_dir, options)

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 515, in process_service_definitions_artifact

create_symlink(src_service_definitions_dir, dest_service_definitions_dir, file, options.force)

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 235, in create_symlink

create_symlink_using_path(src_path, dest_link, force)

File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py", line 247, in create_symlink_using_path

sudo.symlink(src_path, dest_link)

File "/usr/lib/python2.6/site-packages/resource_management/core/sudo.py", line 123, in symlink

os.symlink(source, link_name)

OSError: [Errno 17] File exists

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Super Mentor
@Dmitro Vasilenko

In order to see which symlink creation is causing the issue please try the following:

1. Edit the File "/usr/lib/python2.6/site-packages/ambari_server/setupMpacks.py" and put the following lines just above line 247 and add the "print_info_msg" to print the source and destination symlink paths.

ORIGINAL

244  if force and os.path.islink(dest_link):
245    sudo.unlink(dest_link)
246
247  sudo.symlink(src_path, dest_link)
248  print_info_msg("Symlink: " + dest_link)

.

EDITED

244  if force and os.path.islink(dest_link):
245    sudo.unlink(dest_link)
246
247  print_info_msg("src_path = " + src_path)
248  print_info_msg("dest_link = " + dest_link)
249  sudo.symlink(src_path, dest_link)
250  print_info_msg("Symlink: " + dest_link)

.

Then check the print message

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Contributor

Thanks you very much !

We have next errors, when running (Upgrading database schema)

INFO: Upgrading database schema Error executing query: UPDATE request AS a SET cluster_host_info = b.cluster_host_info FROM stage AS b WHERE a.request_id = b.request_id, errorCode = 0, message = ERROR: column b.cluster_host_info does not exist

Position: 45 23 Jun 2017 12:03:08,078 ERROR [main] SchemaUpgradeHelper:209 - Upgrade failed. org.postgresql.util.PSQLException: ERROR: column b.cluster_host_info does not exist

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Super Mentor

@Dmitro Vasilenko

This looks like an issue with the table schema.

Can you please try this:

1. Please collect latest Ambari DB dump. (to be on safer side)
# pg_dump -U ambari ambari > ambari.sql 

2. Run the following commands:

# psql -U ambari ambari
Password for user ambari: bigdata

3. After logging into the Database terminal, Edit the following table by running the query.

ALTER TABLE stage ADD COLUMN cluster_host_info bytea not null default '{}';

3. Continue your upgrade/installation.

.

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Explorer

@dmitro vasilenko, Were you able to get past the problem?

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Super Mentor

@Sarnath K

An issue with similar stack trace is fixed as part of JIRA: https://issues.apache.org/jira/browse/AMBARI-21263

Which version of ambari are you using ?

Can you please try on Ambari 2.5.2

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Explorer

@Jay Kumar SenSharma I need to upgrade to 2.5.1 from 2.4.2

I need 2.5.1 as it supports HDF 3.x

Ambari 2.5.2 does not support HDF 3.x .... So, how do we resolve this paradox?

Thanks for the answer much. Appreciate that.

Re: Ambari-server upgrade from hdf-2.0 to hdf-3.0

Explorer

@Jay Kumar SenSharma

Also, we have not installed HDPSearch mpack (or) at least, we have not consciously done.. If it was installed without our notice, is there a way to check whethre HDPSearch is causing the issue?