Ambari settings after database restore

[#1]

I had to restore an Ambari database backup, which was a bit behind the current cluster status. Will the Ambari restore "rollback" config changes upon cluster startup, and if so is there a way to "refresh" Ambari with what's current in the cluster. Thank you.

[#2]

@Brenden Cobb

Make sure you backup existing db before the restore

Restore the old db

Restore the backup done in step 1 in different instance and try to match the content

Or

You can export the blueprint of existing cluster and restore , export the blueprint after restore and match the differences

Change the configs as per the differences

[#3]

Thanks. For 1) This sounds like Ambari will overwrite existing component configs when using a database restore? For 2) is the blueprint created from Ambari's settings or the cluster's configs? I'm just not clear whether Ambari will overwrite what's in the cluster or pull, and if there's an option to control this behavior.

[#4]

@Brenden Cobb

1) Yes

2) If you restore db then ambari will push configs based on configs in db. We need to be creative to find the delta.

3) curl -u admin:admin -i -H 'X-Requested-By: ambari' -X GET http://ambariserver:8080/api/v1/clusters/clustername?format=blueprint

This will give you configs from the existing cluster and then run again after db restore

[#5]

Ah, ok. So execute before starting cluster services to get what's there, execute again after starting services and basically do a diff between the 2? This is good.

[#6]

@Brenden Cobb Yes. I believe you have restore the old db ..if you have not then I would run the blueprint statement now and save it ...then restore db , run blueprint command ..

[#7]

@Brenden Cobb Let's connect on linkedin and we can discuss this in detail.

[#8]

Thanks Neeraj, look forward to connecting.

[#9]

@Brenden Cobb Did you try the above solution? I am looking forward to see the final outcome.

[#10]

As suggested, I have the blueprint which reflects all the current information for our stack (2.3.2). When attempting to start services on the cluster Ambari favors 2.2, the previously installed version. My assumption is the Ambari database needs some adjusting to properly recognize 2.3.2. I've tried a few updates to set the installed stack to 2.3.2, but Ambari is unsuccessful at starting the cluster components.

[#11]

@Brenden Cobb Are you upgrading ambari? Whats the previous and latest version?

http://docs.hortonworks.com/HDPDocuments/Ambari-2.2.0.0/bk_upgrading_Ambari/content/_ambari_upgrade_...

2.3.2 is HDP version

2.2.0 is Ambari version

[#12]

Nope, not upgrading anything. Ambari database stack options:

select * from stack;
 stack_id | stack_name |  stack_version
----------+------------+-----------------
        1 | HDP        | 2.3
        2 | HDP        | 2.1.GlusterFS
        3 | HDP        | 2.2
        4 | HDP        | 2.1
        5 | HDP        | 2.0
        6 | HDP        | 2.0.6
        7 | HDP        | 2.3.GlusterFS
        8 | HDP        | 2.0.6.GlusterFS

[#13]

Also:

$ ls /usr/hdp
2.2.0.0-2041  2.2.4.2-2  2.3.2.0-2950  current

Where current is correctly pointing to 2.3.x

[#14]

@Brenden Cobb Are you doing this in production? We need to escalate this as p1 case so that someone from support can help you. HDP 2.2 vs. 2.3 under current...This is not good

[#15]

@Brenden Cobb

since this is after-effect, I'd open a ticket with support. In my experience, i would backup all configs on every node, then try to restart agent on one node at a time as agent will advertise current config on a node to ambari server. Once you confirm everything is restored for that node, you can go to next node.

[#16]

Thanks I will keep that in mind as I progress. Support is not very helpful at the moment..that's why dropped by.

[#17]

@Brenden Cobb which region are you in? You can tail the Ambari server log for configbstatus. @Mark Herring

[#18]

I'm in the Northeast. Thank you for the tips.

[#19]

@Brenden Cobb no problem, I escalated your issue to get proper attention. I'm in Northeast, if you have similar issues, you can reach out to me directly. Also, please post your current status as this thread also needs to know the solution :).

[#20]

@Brenden Cobb please escalate your case to level 1.

[#21]

Thank you Artem. My current status is up in response to Neeraj's comments.