question Re: Ambari settings after database restore in Support Questions

Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 03:26:26 GMT

I had to restore an Ambari database backup, which was a bit behind the current cluster status. Will the Ambari restore "rollback" config changes upon cluster startup, and if so is there a way to "refresh" Ambari with what's current in the cluster. Thank you.

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 03:42:46 GMT

@Brenden Cobb

Make sure you backup existing db before the restore

Restore the old db

Restore the backup done in step 1 in different instance and try to match the content

You can export the blueprint of existing cluster and restore , export the blueprint after restore and match the differences

Change the configs as per the differences

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 04:23:11 GMT

Thanks. For 1) This sounds like Ambari will overwrite existing component configs when using a database restore? For 2) is the blueprint created from Ambari's settings or the cluster's configs? I'm just not clear whether Ambari will overwrite what's in the cluster or pull, and if there's an option to control this behavior.

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 04:28:05 GMT

@Brenden Cobb

1) Yes

2) If you restore db then ambari will push configs based on configs in db. We need to be creative to find the delta.

3) curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET http://ambariserver:8080/api/v1/clusters/clustername?format=blueprint

This will give you configs from the existing cluster and then run again after db restore

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 04:57:18 GMT

Ah, ok. So execute before starting cluster services to get what's there, execute again after starting services and basically do a diff between the 2? This is good.

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 05:10:09 GMT

@Brenden Cobb Yes. I believe you have restore the old db ..if you have not then I would run the blueprint statement now and save it ...then restore db , run blueprint command ..

Re: Ambari settings after database restore

aervits — Mon, 15 Feb 2016 08:09:36 GMT

@Brenden Cobb

since this is after-effect, I'd open a ticket with support. In my experience, i would backup all configs on every node, then try to restart agent on one node at a time as agent will advertise current config on a node to ambari server. Once you confirm everything is restored for that node, you can go to next node.

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 10:05:35 GMT

Thanks I will keep that in mind as I progress. Support is not very helpful at the moment..that's why dropped by.

Re: Ambari settings after database restore

aervits — Mon, 15 Feb 2016 10:30:42 GMT

@Brenden Cobb which region are you in? You can tail the Ambari server log for configbstatus. @Mark Herring

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 10:54:37 GMT

@Brenden Cobb Let's connect on linkedin and we can discuss this in detail.

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 13:40:41 GMT

I'm in the Northeast. Thank you for the tips.

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 13:42:32 GMT

Thanks Neeraj, look forward to connecting.

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 20:19:20 GMT

@Brenden Cobb Did you try the above solution? I am looking forward to see the final outcome.

Re: Ambari settings after database restore

aervits — Mon, 15 Feb 2016 21:53:27 GMT

@Brenden Cobb no problem, I escalated your issue to get proper attention. I'm in Northeast, if you have similar issues, you can reach out to me directly. Also, please post your current status as this thread also needs to know the solution :).

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 23:47:37 GMT

As suggested, I have the blueprint which reflects all the current information for our stack (2.3.2). When attempting to start services on the cluster Ambari favors 2.2, the previously installed version. My assumption is the Ambari database needs some adjusting to properly recognize 2.3.2. I've tried a few updates to set the installed stack to 2.3.2, but Ambari is unsuccessful at starting the cluster components.

Re: Ambari settings after database restore

aervits — Mon, 15 Feb 2016 23:49:09 GMT

@Brenden Cobb please escalate your case to level 1.

Re: Ambari settings after database restore

bcobb — Mon, 15 Feb 2016 23:49:56 GMT

Thank you Artem. My current status is up in response to Neeraj's comments.

Re: Ambari settings after database restore

nsabharwal — Mon, 15 Feb 2016 23:52:33 GMT

@Brenden Cobb Are you upgrading ambari? Whats the previous and latest version?

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_upgrading_Ambari/content/_ambari_upgrade_guide.html

2.3.2 is HDP version

2.2.0 is Ambari version

Re: Ambari settings after database restore

bcobb — Tue, 16 Feb 2016 00:01:06 GMT

Nope, not upgrading anything. Ambari database stack options:

select * from stack;
 stack_id | stack_name |  stack_version
----------+------------+-----------------
        1 | HDP        | 2.3
        2 | HDP        | 2.1.GlusterFS
        3 | HDP        | 2.2
        4 | HDP        | 2.1
        5 | HDP        | 2.0
        6 | HDP        | 2.0.6
        7 | HDP        | 2.3.GlusterFS
        8 | HDP        | 2.0.6.GlusterFS

Re: Ambari settings after database restore

bcobb — Tue, 16 Feb 2016 00:16:42 GMT

Also:

$ ls /usr/hdp
2.2.0.0-2041  2.2.4.2-2  2.3.2.0-2950  current

Where current is correctly pointing to 2.3.x