Member since
11-14-2015
268
Posts
122
Kudos Received
29
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2525 | 08-07-2017 08:39 AM | |
3951 | 07-26-2017 06:06 AM | |
9364 | 12-30-2016 08:29 AM | |
7372 | 11-28-2016 08:08 AM | |
6989 | 11-21-2016 02:16 PM |
09-13-2017
07:18 AM
CompactionState.NONE : means no compaction is currently running. bq. And major compaction is an asychronous process, how to predict the wait time to get the state form the client API You need to keep on polling the API with some arbitrary wait time.
... View more
09-13-2017
06:15 AM
bq. 1) How to figure out the completion status for the compaction, since hbase client api majorCompact is an Asynchorous process you can use below API. CompactionState compactionState = admin.getCompactionState(table.getName()); 2) Is it mandatory to wait until compaction process completion , to query hbase for real time process Check your resources consumption with compaction as it impacts I/O, CPU usage and network . In standard server configuration , it is fine to run real time process during compaction.
... View more
09-07-2017
09:58 AM
1 Kudo
It's better you declare every field as VARCHAR and then use functions to convert them to numbers[1] for mathematical operations. [1] https://phoenix.apache.org/language/functions.html#to_number
... View more
08-10-2017
11:21 AM
Do you have any errors in regionserver logs (it seems hbase:namespace table is not getting assigned some how).
... View more
08-09-2017
03:04 PM
Before running above command please take regionserver and master down if not already.(and keep the zookeeper running)
... View more
08-09-2017
02:39 PM
If it is not a production cluster(or not doing replication or other stuff dependent on zookeeper), can you try cleaning your zookeeper as it may be possible that there is znode for non-existent tables . bin/hbase clean --cleanZk
... View more
08-07-2017
10:06 AM
Map intermediate data will be written and sorted on local disk before sending to the reducer machines. You can reduce Map output Use Combiner in between Compressing it with Gzip to save network IO but there will be a tradeoff for CPU (mapred.compress.map.output=true, mapred.map.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec) Decrease split size(this will distribute map across server) and increase the number of reducers so that they have fewer amount of data to sort and process stop speculative execution( mapred.map.tasks.speculative.execution=false) If you can optimize on sorting, update algorithm for map.sort.class bq. Will i get any performance improvement if i increase io.sort.mb paramter when Map() task generates huge amount of data? Yes (but impact may not be huge), you can use with io.sort.factor
... View more
08-07-2017
08:41 AM
can you paste the complete stack trace.
... View more
08-07-2017
08:39 AM
Currently, It seems there is no option in distcp to do so. File can be replaced by a File and directory by a directory by passing "-update" option to command.
... View more
08-07-2017
06:22 AM
you can use "major_compact" command to run a major compaction on the table. In HBase shell:- hbase(main):013:0> major_compact 'tablename'
... View more