Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP 2.6 upgrade

HDP 2.6 upgrade

New Contributor

I Upgrade ambari to 2.6 and hdp from 2.4 to 2.6

Post ambari upgrade, I restarted ambari-server few times before I jump into HDP2.6 upgrade. It was working fine.

Both the upgrade went well. Post HDP upgrade (to2.6) when I restart ambari server, it does not come up.

Log: Attached as a txt file

26 May 2018 19:24:01,827 INFO [main] StackServiceDirectory:129 - No repository information defined for , serviceName=SQOOP, repoFolder=/var/lib/ambari-server/resources/stacks/HDP/2.3.ECS/services/SQOOP/repos 26 May 2018 19:24:01,828 INFO [main] StackServiceDirectory:129 - No repository information defined for , serviceName=KNOX, repoFolder=/var/lib/ambari-server/resources/stacks/HDP/2.3.ECS/services/KNOX/repos 26 May 2018 19:24:01,830 INFO [main] StackDirectory:546 - Role command order info was loaded from file: /var/lib/ambari-server/resources/stacks/HDP/2.3.ECS/role_command_order.json 26 May 2018 19:24:01,838 INFO [main] StackDirectory:489 - Stack '/var/lib/ambari-server/resources/stacks/HDP/2.2' doesn't contain an upgrade directory

26 May 2018 19:24:04,981 INFO [main] StackContext:250 - Loaded all VDF in 378ms

26 May 2018 19:24:05,006 INFO [main] StackManager:252 - Creating links

26 May 2018 19:24:05,009 INFO [main] ActionDefinitionManager:130 - Added custom action definition for check_host

26 May 2018 19:24:05,009 INFO [main] ActionDefinitionManager:130 - Added custom action definition for update_repo

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for clear_repocache

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for validate_configs

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for install_packages

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for ru_execute_tasks

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for remove_previous_stacks

26 May 2018 19:24:05,010 INFO [main] ActionDefinitionManager:130 - Added custom action definition for force_remove_packages

26 May 2018 19:24:05,018 INFO [main] UpdateActiveRepoVersionOnStartup:80 - Updating existing repo versions with service repos.

26 May 2018 19:24:05,061 INFO [main] RepoUtil:136 - Added 0 service repos: [] 26 May 2018 19:24:05,062 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,062 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,062 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,063 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,063 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,063 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,064 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,064 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,064 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,064 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,065 INFO [main] RepoUtil:136 - Added 0 service repos: []

26 May 2018 19:24:05,067 ERROR [main] AmbariServer:1074 - Failed to run the Ambari Server org.apache.ambari.server.AmbariException: An error occured during updating current repository versions with stack repositories. at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:99) at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128) at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1062) Caused by: java.lang.NullPointerException at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:87) ... 2 more

Thanks in advance for your help....

10 REPLIES 10

Re: HDP 2.6 upgrade

New Contributor

Latest update,

I found cluster_version not present in ambari db. I have created and added entry there.

still no luck.

Error:

27 May 2018 16:46:03,274 INFO [main] ServiceModule:185 - Resolve service 27 May 2018 16:46:03,274 INFO [main] ServiceModule:198 - Display name service/parent: null/Pig

27 May 2018 16:46:03,298 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:94 - Loading latest URL info from http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.3.GlusterFS,HDP-2.3.ECS,HDP-2.5,HDP-2.6,HDP-2.3,HDP-2.4,HDP-2.1.GlusterFS,HDP-2.1,HDP-2.2,HDP-2.0

27 May 2018 16:46:03,818 INFO [Stack Version Loading Thread] RepoUrlInfoCallable:110 - Loaded URI http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json for stacks HDP-2.3.GlusterFS,HDP-2.3.ECS,HDP-2.5,HDP-2.6,HDP-2.3,HDP-2.4,HDP-2.1.GlusterFS,HDP-2.1,HDP-2.2,HDP-2.0 in 521ms

27 May 2018 16:46:03,823 INFO [main] StackContext:208 - Loaded urlinfo in 529ms

27 May 2018 16:46:04,165 INFO [Stack Version Loading Thread] RepoVdfCallable:130 - Stack HDP-2.3 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null

27 May 2018 16:46:04,171 INFO [Stack Version Loading Thread] RepoVdfCallable:130 - Stack HDP-2.4 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null

27 May 2018 16:46:04,331 INFO [main] StackContext:250 - Loaded all VDF in 507ms

27 May 2018 16:46:04,353 INFO [main] StackManager:252 - Creating links

27 May 2018 16:46:04,355 INFO [main] ActionDefinitionManager:130 - Added custom action definition for check_host

27 May 2018 16:46:04,355 INFO [main] ActionDefinitionManager:130 - Added custom action definition for update_repo

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for clear_repocache

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for validate_configs

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for install_packages

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for ru_execute_tasks

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for remove_previous_stacks

27 May 2018 16:46:04,356 INFO [main] ActionDefinitionManager:130 - Added custom action definition for force_remove_packages

27 May 2018 16:46:04,363 INFO [main] UpdateActiveRepoVersionOnStartup:80 - Updating existing repo versions with service repos.

27 May 2018 16:46:04,413 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,413 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,414 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,414 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,414 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,415 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,415 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,415 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,415 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,416 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,416 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,416 INFO [main] RepoUtil:136 - Added 0 service repos: []

27 May 2018 16:46:04,419 ERROR [main] AmbariServer:1074 - Failed to run the Ambari Server org.apache.ambari.server.AmbariException: An error occured during updating current repository versions with stack repositories. at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:99) at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128)

Re: HDP 2.6 upgrade

New Contributor

Re: HDP 2.6 upgrade

New Contributor

looks like AMBARI-18467

Re: HDP 2.6 upgrade

New Contributor

I am not sure if this create the problem.... But I am not using ubuntu. I am on RHEL.

27 May 2018 02:55:56,966 INFO [Stack Version Loading Thread] RepoVdfCallable:130 - Stack HDP-2.3 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null

27 May 2018 02:55:56,978 INFO [Stack Version Loading Thread] RepoVdfCallable:130 - Stack HDP-2.4 cannot resolve OS debian6 to the supported ones: suse11,redhat7,debian7,redhat6,ubuntu14,ubuntu12. Family: null

Highlighted

Re: HDP 2.6 upgrade

Contributor

@S Mahapatra

Few questions:

  1. Can you attach the entire ambari-server.log file?
  2. Do you have any mpacks installed through Ambari prior to Upgrade?
  3. Have you enabled any experimental features using the 'experimental' flag?
  4. Please mention exact HDP-2.6 and Ambari-2.6 versions that you upgraded to

Re: HDP 2.6 upgrade

New Contributor

hi @vsharma

I have posted the versions and log.

Re: HDP 2.6 upgrade

New Contributor

hi @vsharma

Thanks for your your interest. Please find the reply bellow:

ambari-server-28-log.txt --- ambari server log.

No mpacks. I though mpacks is for HDF. I have not installed. Do we need it?

I have not enabled any experimental feature. Post upgrade to hdp-2.6.4.0 from HDP 2.4, I have installed Spark2, Zeppline and enable Interactive hiveserver2 and everything went well fine.

I upgrade to ambari 2.6.1.5

76428-28-ambari.jpg

restarted ambari-server few time . it was working fine.

then started the upgrade (rolling) following the process and it went well. hdf-2.4.2 to hdp2.6.4

76429-28hdpversion.jpg

76430-28-hdp-rolling-upgrade.jpg

76431-28-hdp-current-post-upgrade.jpg

ambari db is on mysql 5.6 and mysql-java-connector jar version is 5.1.47

post hdp upgrade when I restart ambari server it is not starting.

--------

ERROR [main] AmbariServer:1078 - Failed to run the Ambari Server

org.apache.ambari.server.AmbariException: An error occured during updating current repository versions with stack repositories. at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:99)

at org.apache.ambari.server.orm.AmbariJpaLocalTxnInterceptor.invoke(AmbariJpaLocalTxnInterceptor.java:128) at org.apache.ambari.server.controller.

AmbariServer.main(AmbariServer.java:1066) Caused by: java.lang.NullPointerException at org.apache.ambari.server.stack.UpdateActiveRepoVersionOnStartup.process(UpdateActiveRepoVersionOnStartup.java:87)

... 2 more

After this problem I have upgraded ambari to 2.6.2.0 still same issue.

Re: HDP 2.6 upgrade

Super Guru
@S Mahapatra

I did not read entire comments but based on error stack it looks like you have some problem with mpack installation.

Can you please follow below steps and let me know how it goes:

  1. Take a backup of the following folder content to different directory:
    /var/lib/ambari-server/resources/common-services/REGISTRY
    /var/lib/ambari-server/resources/common-services/NIFI
  2. Clear the contents from the above path.
  3. Reinstall the mpack.

Re: HDP 2.6 upgrade

New Contributor

No NiFi installed on ambari server nor any other node through ambari. THe above 2 directory not existing.

mpack, I guess related to HDF not HDP. Shall I install it manually?