This check seems to be added from https://issues.apache.org/jira/browse/HDFS-8965   /data01/dfs/jn/pt-nameservice/current/edits_inprogress_0000000000182289435 is either corrupted, or perhaps It is a log from before an upgrade?   Either way, To get it back I would suggest backing up the /data01/dfs/jn/pt-nameservice/current/ directory somewhere else and copy the journalnode data from one of your other journalnodes to that location. ... View more

So am I understanding correctly?   Data generating server -> HTTP Post -> Flume HTTP Source -> Flume sink -> etc.   and you want to make two Flume HTTP Source machines that can be written to and be able to recieve that data, in case one of them went down?  You also don't want to have to manage something like a load balancer/proxy in between the Data generating server and the Flume HTTP Source box?   If you can handle de-duplication on your back-end, then I think you could do this by sending the same data to two different Flume HTTP Source servers at the same time, possibly tagging the data in your sink to help you de-duplicate later.     Data generating server -> HTTP Post -> Flume HTTP Source - Flume sink with tag --> to de-duplication                |                -------------------->  HTTP Post -> Flume HTTP Source - Flume sink with tag --> to de-duplication ... View more

doing some quick searching, this blog seems to be doing what I think is your intent, taking logs, storing in kafka, distributing to various consumers, one of those consumers being Cloudera Search (solr) [1]  you could make it simpler and store directly to solr[2] if you aren't planning on consuming the same data from multiple sources.  instead of logstash you could also use Flume [3] [4] as well.    [1]https://www.elastic.co/blog/logstash-kafka-intro [2]https://github.com/lucidworks/solrlogmanager [3]http://www.cloudera.com/documentation/archive/search/1-3-0/Cloudera-Search-User-Guide/csug_flume_solr_sink.html [4]http://blog.cloudera.com/blog/2014/11/flafka-apache-flume-meets-apache-kafka-for-event-processing/ ... View more

So this is 13 days late, so I'm imagining you already have a solution or have moved on. Commenting for future searchers.    As you have learned the hard way, the only "safe" hbck option is the -fixAssignments option. every other option is potentially dangerous. Having said that, I've experienced -fixAssignments multiply assign regions if there are regions in transition. this can be fixed by running -fixAssignments again, or by failing over the HMaster and allowing the assignment manager to fix it.    Unfortunately there isn't a single way that the -repair option could have trashed things. There are 3 "brains" for Hbase, there is the data in HDFS under /hbase for each region in the .regioninfo file, the zookeeper data, and the Hbase .META. region.  Depending on which one having a problem will determine which option is the proper one to get fixed. When having an issue, The first step is to figure out which one of these is incorrect. This is a deep topic so i've added some helpful links to read more on this[1][2]   Based on all of your tables disappearing, but presumably being able to read it before I would assume that There were holes detected for basically all tables, and detected from a previous region state with different splits. hbase decided to fix it by filling those "holes" with empty regions, deleting the existing regions.    To recover from that, You would shut down hbase, move the table files from .Trash in hdfs back to the original location and do an offline meta repair. hbase org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair. From there you bring up hbase and troubleshoot any issues from there.  Again, This would only be proper based on the assumption above being correct, and would absolutely not be the proper action if HDFS were corrupt and it was Meta that was correct.      [1]http://hbase.apache.org/book.html#_region_overlap_repairs [2]http://www.cloudera.com/documentation/enterprise/5-4-x/topics/admin_hbck_poller.html   ... View more