Well, I did clean everything as suggested, double checked configuration you mentionned, but still the same problem occurs. The result of the ps command returns this : [root@sandbox ~]# ps -ef | grep ^ams ams 14734 1 0 19:38 ? 00:00:00 bash /usr/lib/ams-hbase/bin/hbase-daemon.sh --config /etc/ams-hbase/conf foreground_start master ams 14748 14734 10 19:38 ? 00:00:26 /usr/lib/jvm/java/bin/java -Dproc_master -XX:OnOutOfMemoryError=kill -9 %p -XX:+UseConcMarkSweepGC -XX:ErrorFile=/var/log/ambari-metrics-collector/hs_err_pid%p.log -Djava.io.tmpdir=/var/lib/ambari-metrics-collector/hbase-tmp -Djava.library.path=/usr/lib/ams-hbase/lib/hadoop-native/ -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/ambari-metrics-collector/gc.log-201706141938 -Xms1536m -Xmx1536m -Xmn256m -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -Dhbase.log.dir=/var/log/ambari-metrics-collector -Dhbase.log.file=hbase-ams-master-sandbox.hortonworks.com.log -Dhbase.home.dir=/usr/lib/ams-hbase/ -Dhbase.id.str=ams -Dhbase.root.logger=INFO,RFA -Dhbase.security.logger=INFO,RFAS org.apache.hadoop.hbase.master.HMaster start And ambari-metrics-collector.log (cf attachment ambari-metrics-collector.zip) file contains several errors. The latest ones are : 2017-06-14 19:39:29,423 ERROR org.apache.helix.controller.GenericHelixController: ClusterEventProcessor failed while running the controller pipeline java.lang.NullPointerException at org.apache.helix.controller.GenericHelixController.handleEvent(GenericHelixController.java:276) at org.apache.helix.controller.GenericHelixController$ClusterEventProcessor.run(GenericHelixController.java:595) 2017-06-14 19:39:29,430 ERROR org.apache.helix.controller.GenericHelixController: Cluster manager: sandbox.hortonworks.com is not leader. Pipeline will not be invoked Any clue about this ? Could it be the reason why AMS automatically stops ? Thanks ... View more

@Jay Thanks a lot for your quick answer ! I checked the property you mentionned and it actually has the "embedded" value => I stopped HBase service on the sandbox. I also stopped Flume, Hive, Spark2, Knox, but still the same behavior : metrics collector starts, and does down after a while...Is there a particular place (logs...) to check if my VM runs out of memory ? (I dedicated 4 cores and 16GB for the sandbox, but dont know how much ambari metrics expects to work...) Your comments also raise an additional question : in distributed mode, AMS "embedded hbase" writes metrics to HDFS - instead of local disk. Does it means that AMS never uses the standard HBase service of the cluster, but only an embedded hbase, or is there a mean to change this somehow ? Thanks again ... View more

In fact, was my fault : when my box went back from stand-bye mode, I had not restarted hbase ... It works now 🙂 ... View more

Thanks for your help, I got rid with python error. Anyway, I still cannot connect from my python client to hbase thrift server (about your question, I'm trying to use a vanilla HBase, without any other component), because I can see following error on server side : 2017-05-30 08:01:11,816 WARN [thrift-worker-0-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connexion refusée at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2017-05-30 08:01:11,916 ERROR [thrift-worker-0] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2017-05-30 08:01:11,917 WARN [thrift-worker-0] zookeeper.ZKUtil: hconnection-0x3330972d0x0, quorum=localhost:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:419) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:905) It seems that HBase is trying to reach zookeeper, but I didn't setup any zookeeper...Is it possible to run HBase without zookeeper ? Thanks again Regards ... View more

After more reading, it seems that region replication may be used for read high availability... If I understand properly, it means that when a RS fails, its regions are moved to other "valid" region servers and are still available, but it may take a while ... So region replication's purpose is just to reduce this waiting period ? Nothing related to data physical replication in order to guarantee that we won't loose any data, right ? ... View more

Among the things I gave a try to, I suspected that maybe parallelizing the whole set of file paths (more than 100000) one shot without specifying the number of partitions was a problem, because it seemed that some containers were assigned very few files whereas others had many many files to process, and thus some containers were not used after a while, waiting after the others to complete their tasks... For that reason, I tried to reduce the number of files to distribute (in my driver application, I added a loop to "parallelize" those files by "batch" of 1000 files, so that every 1000 files a fair distribution is done to obtain better repartition over my containers, but couldnt understand exactly how spark proceeded. Basically, for each iteration of my driver, I have 1000 files to process. I arbitrarily splitted these files in 100 partitions => spark has 100 tasks to complete, each one focusing on 10 files. If I ask for 20 executors with 2 cores per executors (for example, and supposing that yarn actually provides me such resources), it means that spark should be able to "distribute" 40 tasks in parallel, and keep 60 tasks to feed again the containers that would first complete their task...Am I right ? But one additional detail that may be important : it seems that first tasks are "pretty fast" (almost as fast as non-spark implementation), but subsequent ones are getting slower and slower along time, till a new batch of 1000 files is started ... View more