Anpan K I can understand the confusion that's brewing in your mind. In a kerberized production cluster, you'd usually have Edge node, Master and slave nodes. I will not go in the description and placement of every single component but the below distribution gives you a picture. Note: The worker node usually MUST have at least 2 slave processes Datanode & NodeManager, all the client software goes on the Edge node and the Master node holds the other components notably NameNode,RM,Zookeeper HA's etc Master (3x) Namenode YARN (RM) Zookeeper HS2 ..... ..... Hbase Master Slave(worker nodes) n DataNodes NodeManager Region Servers Edge Node Knox ZK Client HDFS Client MR Cllient ... ... YARN client Below on the knowGateway is installed all the client software, the Hadoop services here represent the Master and Slaves nodes. The Knox gateway should sit on the Edge node as should be the only access to the cluster as illustrated above. HTH ... View more

@karthik nedunchezhiyan This potentially happens JournalNodes when one of the nodes is lagging behind the others (eg because its local disk is slower or there was a network blip), it receives edits after they've been committed to a majority. It can tell this because the committed txid included in the request info is higher than the highest txid in the actual batch to be written. In this case, we know that this batch has already been fsynced to a quorum of nodes, so we can skip the fsync() on the laggy node, helping it to catch back up. The Active NameNode will write/read edits to the below URI, which is a shared address by the JournalNodes and provides the shared edits storage, it's ONLY written to by the Active nameNode and read by the Standby NameNode to stay up-to-date with all the file system changes the Active NameNode makes. Though you must specify several JournalNode addresses, you should only configure one of these URIs. dfs.namenode.shared.edits.dir QuorumJournalManager is responsible for syncing the missing transactions On a journal node, the missing transaction is recovered by the TransferFsImage class from another journal node thats up to date in this case journalnodes (1 and 3) Started a 3-node QJM cluster strace -efdatasync,write -f <pid of one JN> Write some txns to the NN it will show a lot of fdatasync and write calls. kill -STOPped that JN for 10-15 seconds kill -CONT that JN You will see a bunch of write() with no fdatasync calls while it was still catching up. After it caught up, it started syncing again. ... View more

@Praveen Atmakuri I want to ensure the document in the link above will move your on-premise MySQL to MySQL in Azure.Azure Database for MySQL is a Microsoft cloud based service based on the MySQL Community Edition database engine. Advantages Built-in high availability with no additional cost. Predictable performance, using inclusive pay-as-you-go pricing. Scale as needed within seconds. Secured to protect sensitive data-at-rest and in motion. Automatic backups and point-in-time-restore for up to 35 days. Enterprise-grade security and compliance. Difference between Azure MSSQL and Azure MySQL DB MySQL is open source while MSSQL is licensed(commercial). MySQL supports more programming languages than MSSQL MySQL supported several Platforms Windows, Linux and Mac OS while MSSQL runs exclusively on windows though Microsoft recently announced MSSQLwill be available on Linux. MySQL supports a number of storage engines. While using MySQL even have option to use a plug-in storage engine. MySQL does not allow users to kill or cancel a query when it is running But SQL Server programmers can truncate a database query during execution without killing the entire process. These are just but a few distinct differences between the 2 RDBMS's If you found this answer addressed your question, please take a moment to log in and click the "accept" link on the answer. ... View more