Member since
07-08-2013
35
Posts
11
Kudos Received
6
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5762 | 08-05-2015 08:18 AM | |
2353 | 06-18-2015 04:33 PM | |
16095 | 05-10-2015 08:10 PM | |
51689 | 05-10-2015 07:34 PM | |
5117 | 05-08-2015 09:09 AM |
04-02-2019
07:48 AM
Hi Vinod, Can you please start a different thread and share your Master logs with us? JMS
... View more
09-26-2018
06:11 AM
1 Kudo
Hi Ayush, Have you tied to run hdfs fsdk / ? JM
... View more
04-07-2017
04:48 AM
It depends what you want to achieve, right? If you goal is to repair your production cluster, then it might be relevant. If you goal is just to test and see what it does, then what ever command it is, it should not be tested in production? Basically, the command just does what I described. It deletes everything. It's you to decide if it's relevent for your production environment or not. If you goal is just to drop a table, or all tables, then the shell is better. Are you facing any issue with your production environment?
... View more
04-07-2017
04:23 AM
Hi Sandeep, This command will remove ALL HBase data... Snapshots, tables, WALs, etc. So be very careful with it. JMS
... View more
03-31-2017
03:09 PM
Hi, Your syntax seems to be incomplete. Where do you detail which HBase table your Hive table should be map to? Usually this is done using the TBLPROPERTIES field... JMS
... View more
08-05-2015
08:18 AM
Hi Asif, You will need to do some code for that. Create a small table, put some data in it and call the bulk delete. Here is an example: https://github.com/apache/hbase/blob/master/hbase-examples/src/test/java/org/apache/hadoop/hbase/coprocessor/example/TestBulkDeleteProtocol.java You might be able to re-use most of this code. JM
... View more
08-05-2015
08:07 AM
1 Kudo
Hi Asif, Bulk Delete is an EndPoint coprocessor. Therefore, it is not related to a specific table, but more loaded for a RegionServer. Because of that, you will not see it when you will describe a table from the shell because it is not attached to that specific table. Only observers are attached to the tables and are displayed in the describe command. HTH, Jean-Marc
... View more
07-07-2015
06:34 AM
Hi Siddesh, This is not mention, but it's a potential cause of the under replicated blocks. Can also be anything else. JMS
... View more
07-07-2015
06:23 AM
Hi Siddesh, Adding a note is not related to the under-replicated blocks. under-replicated blocks where most probably already there before the new node got added. By default, for the JARs, MR set the replication to 10. So if there is less than 10 nodes in the cluster you will most always have under replicated blocks. You should check with fsck to see what is missing. Balancer will not help for that. JM
... View more
06-18-2015
04:33 PM
1 Kudo
Hi Neil, As you said in your message, it all depends or what you want to run, however here are some guidelines. I'm not talking about the "data" drives since you can have as many as you want with the size you need. 1) CM requirements will apply to CDH, like for the /var folder 2) CM will start to alert you when Journal Nodes, Namenodes, and other processes directories start to be under 10GB. Therefore, accounting at least 20GB per service for the "meta" (logs, configs, binaries, etc.) is a good idea. So if you have a YARN + DN + Spark on a node, give them at least 60GB of disk space for 3) Master processes will use space based on the size of the cluster. Indeed, the bigger the cluster is, the more data, the more blocks, the more space is used on the NN and JN directories. So for clusters bigger than 30 nodes you might want to think about giving them a bit more. Now. It is not recommended to run any sercive on the OS disk (and not just partition). And since disks are bigger and bigger, you might end up with something like 1TB available on your partitition sur CM agaent + CDH services (on worker nodes). If that's the case, I don't think you should really worry about the available space and just share this space between the different mounting points (if split in partitions). Let me know if I can provide anymore details or information or if this doesn't reply to your question. JM
... View more