HA for History Server and App Timeline Server

[#1]

Do we support HA for History and Timeline Server? if either of these services is down, is there any impact to hadoop services? what is the impact if they are down for couple of hours. please advise.

[#2]

@rbalam

See this Source

1. The history can be stored in memory or in a leveldb database store; the latter ensures the history is preserved over Timeline Server restarts.
2. The single-server implementation of the Timeline Server places a limit on the scalability of the service; it also prevents the service being High-Availability component of the YARN infrastructure.

[#3 · Accepted solution]

Unfortunately, this is one of the remaining Yarn components that does not support HA at the moment. However there are already plans for a new Timeline Server (v2), which will be more scalable and reliable.

If your Timeline Server is unavailable the client will retry to publish the application data a couple of times before its giving up. This can be configured using "yarn.timeline-service.client.max-retries" (defaults to 30)

Check out this page https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/TimelineServer.html

[#4]

@Neeraj Sabharwal

and @Jonas Straub

There is a follow up question on Timeline and History Server .. Could you please take a look at this and let me know your input on how best we can answer this issue ..

Client Says:

"So how would loosing Timeline server impact YARN (or any other services)? Does it have any impact on ResourceManagers or the map reduce jobs? I have the same question about History Sever.

And would it be possible to install and configure Timeline server on two nodes in the cluster and store their data in a replicated file system (via drbd)? In this case the clients will be using a floating IP address in order to connect to the active Timline server."

[#5]

@rbalam

Based on this

"The history can be stored in memory or in a leveldb database store; the latter ensures the history is preserved over Timeline Server restarts.The single-server implementation of the Timeline Server places a limit on the scalability of the service; it also prevents the service being High-Availability component of the YARN infrastructure."

You can research on http://leveldb.org/ and see if you want to replication. I don't think this all is supported.

[#6]

The installation via blueprint won't allow you to create a second timeline server. (It fails suggesting that you can only install it on one.)

If you were to put the leveldb store on an ephemeral mount, you could move the service (and the mount) to a second server managed by another failover method. It isn't pretty, but it is possible.