Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we scale out Atlas server to multiple nodes?

Solved Go to solution
Highlighted

Can we scale out Atlas server to multiple nodes?

New Contributor

I can see Atlas support HA but looks like there will be only one node being active?

I doubt about this because I'm seeing that the "TypeRegistery" in memory is loading when nodes start up, so if there is some one create a new classification type, only the nodes that accept this request will know this change and update it's in memory "TypeRegistery" accordingly. Is my understanding correct?

Thanks,

Eva

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Can we scale out Atlas server to multiple nodes?

From the Apache document: http://atlas.apache.org/HighAvailability.html

Implementation Details of Atlas High Availability

  • The automatic selection of an Active instance, as well as automatic failover to a new Active instance happen through a leader election algorithm.
  • For leader election, we use the Leader Latch Recipe of Apache Curator.
  • The Active instance is the only one which initializes, modifies or reads state in the backend stores to keep them consistent.
  • Also, when an instance is elected as Active, it refreshes any cached information from the backend stores to get up to date.
  • A servlet filter ensures that only the active instance services user requests. If a passive instance receives these requests, it automatically redirects them to the current active instance.

Hope this clears your doubt.

3 REPLIES 3

Re: Can we scale out Atlas server to multiple nodes?

From the Apache document: http://atlas.apache.org/HighAvailability.html

Implementation Details of Atlas High Availability

  • The automatic selection of an Active instance, as well as automatic failover to a new Active instance happen through a leader election algorithm.
  • For leader election, we use the Leader Latch Recipe of Apache Curator.
  • The Active instance is the only one which initializes, modifies or reads state in the backend stores to keep them consistent.
  • Also, when an instance is elected as Active, it refreshes any cached information from the backend stores to get up to date.
  • A servlet filter ensures that only the active instance services user requests. If a passive instance receives these requests, it automatically redirects them to the current active instance.

Hope this clears your doubt.

Re: Can we scale out Atlas server to multiple nodes?

New Contributor

Thank you Sandeep!

To double confirm, there will be only one active node for Atlas?

I'm asking this because, nowadays, we have 2 ways to scale our service's capability: scale up (make the server machine more powerful) or scale out (have more identical instances deployed to handle more request). So base on the current design of Atlas, sounds like it doesn't support the later one?

Thanks,

Eva

Re: Can we scale out Atlas server to multiple nodes?

You are right, Only one node will be active at any point of time. Here the HA is like Active and Standby. If one goes down the other one will become active.

** If this helps addressing your query, Do mark this as 'Accepted'. :)

Don't have an account?
Coming from Hortonworks? Activate your account here