can you try adding a callback to producer to check whether all calls are succeeding or not ProducerRecord<byte[],byte[]> record = new ProducerRecord<byte[],byte[]>("the-topic", "key".getBytes(), "value".getBytes()); producer.send(myRecord, new Callback() { public void onCompletion(RecordMetadata metadata, Exception e) { if(e != null) e.printStackTrace(); System.out.println("The offset of the record we just sent is: " + metadata.offset()); } }); ... View more

check your master logs , search for region which is in transition and get the regionserver info where it is trying to open a region and check the logs of that regionserver. If the logs are normal and you find all your regions (in transition) are failing to open on one regionserver only, then stop it and see if regions open properly ... View more

Yes HBase requires zookeeper cluster for coordinating various tasks. You should have one zookeeper cluster(minimum 3 nodes for high availability). You can choose any nodes for running zookeeper. Once your cluster is ready, you need to update zookeeper quorum in configuration. But I'll still suggest to use ambari which will do installation of zookeeper and configure various component relying on it automatically. ... View more

@biswajit chakroborty, Best way is to use ambari for HBase installation as it will help you in detecting problems with different components involved in HBase (hdfs,zookeeper, hbase). To install through ambari:- https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_Installing_HDP_AMB/content/ch_Installing_Ambari.html Here is the documentation , if you want to manually install hbase from HDP. https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/ch_installing_hbase_chapter.html bq. One more point do I need to install/run zookeeper in master and slave node separately for HBASE.? It is not necessary to have zookeeper local to any node of hbase, if you already have a zookeeper cluster for other services , you can re-use it for HBase also. bq. Present situation, All services are running in master and slave but not able to "list" or create table in Hbase. You should check zookeeper, hbase master and regionserver logs for any exception. ... View more

@Sean Roberts, you can use kerberos authentication instead and configure SASL for encrypting traffic https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_kerberos_setup_hive_view.html you can encrypt the traffic by enabling following property with your kerberos authentication. <property> <name>hive.server2.thrift.sasl.qop</name> <value>auth-conf</value> <description>Sasl QOP value; one of 'auth', 'auth-int' and 'auth-conf'</description> </property> Change the HIVE AUTHENTICATION field as per the above document to add sasl.qop=auth-conf at the end auth=kerberos;principal=hive/_HOST@hortoworks.com;sasl.qop=auth-conf ... View more

@Sean Roberts, Is there any specific reason you are looking for SSL and not using kerberos authentication instead? I can see kerberos authentication documentation.(but not sure if there is any for SSL) https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/section_kerberos_setup_hive_view.html ... View more

It is enabled by default(FlushLargeStoresPolicy). You just need to configure below property. It will not flush those column family who size is less than the size specified in below property hbase.hregion.percolumnfamilyflush.size.lower.bound ... View more