Member since
07-17-2019
738
Posts
433
Kudos Received
111
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 3478 | 08-06-2019 07:09 PM | |
| 3678 | 07-19-2019 01:57 PM | |
| 5208 | 02-25-2019 04:47 PM | |
| 4674 | 10-11-2018 02:47 PM | |
| 1772 | 09-26-2018 02:49 PM |
04-16-2017
01:37 AM
1 Kudo
Use Scan.setBatch(int) to control the number of records fetched per RPC with the Java API. The API call you are making only wraps calls to ResutlScanner.next(). It does not affect the underlying RPCs. You may also have to increase hbase.client.scanner.max.results.size as this caps the numbers of records return in a single RPC (default 2MB). The Thrift and REST servers do NOT cache results. Please disregard the comment which asserts this.
... View more
04-16-2017
01:28 AM
Use `jstack` to identify why the init process is hanging. Most likely you do not have correct accumulo-site.xml or ZooKeeper or HDFS are not running.
... View more
04-13-2017
03:58 PM
Have you performed a sanity check that the RegionServer which gave this error can connect to that IP+port? You can easily use telnet to perform this check (e.g. `telnet 172.16.3.196 16000`). If you get a connection refused error, either the HBase master is not running or there is a network issue preventing this node from talking to the Master.
... View more
04-11-2017
03:21 PM
1 Kudo
You can also use a standard cron implementation via Linux. e.g. echo "major_compact 'FOO'" | hbase shell -n You could schedule the above to run on a specific node at your off-peak time. Be sure to monitor the output so that you can react to any possible failures.
... View more
04-11-2017
04:58 AM
3 Kudos
The data is base64 encoded. This is because the data in HBase might otherwise make invalid XML. Base64 decode the rowkey, column and cell data in your client application.
... View more
04-05-2017
04:27 PM
4 Kudos
The Phoenix Query Server is an HTTP server which expects very specific request data data. Sometimes, in the process of connecting different clients, the various configuration options of both client and server can create confusion about what data is actually being sent over the wire. This confusion leads to questions like "did my configuration property take effect" and "is my client operating as I expect". Linux systems often have a number of tools available for analyzing network traffic on a node. We can use one of these tools, ngrep, to analyze the traffic flowing into the Phoenix Query Server. From a host running the Phoenix Query Server, the following command would dump all traffic from any source to the Phoenix Query Server. $ sudo ngrep -t -d any port 8765 The above command will listen to any incoming network traffic on the current host and filter out any traffic which is not to the port 8765 (the default port for the Phoenix Query Server). A specific network interface (e.g. eth0) can be provided instead of "any" to further filter traffic. When connecting a client to the server, you should be able to see the actual HTTP requests and responses sent between client and server. T 2017/04/05 12:49:07.041213 127.0.0.1:60533 -> 127.0.0.1:8765 [AP]
POST / HTTP/1.1..Content-Length: 137..Content-Type: application/octet-stream..Host: localhost:8765..Connection: Keep-Alive..User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_45)..Accept-Encoding: gzip,deflate.....?org.apache.calcite.avatica.proto.Requests$OpenConnectionRequest.F.$2ba8e796-1a29-4484-ac88-6075604152e6....password..none....user..none
##
T 2017/04/05 12:49:07.052011 127.0.0.1:8765 -> 127.0.0.1:60533 [AP]
HTTP/1.1 200 OK..Date: Wed, 05 Apr 2017 16:49:07 GMT..Content-Type: application/octet-stream;charset=utf-8..Content-Length: 91..Server: Jetty(9.2.z-SNAPSHOT).....Aorg.apache.calcite.avatica.proto.Responses$OpenConnectionResponse......hw10447.local:8765
## The data above is in ProtocolBuffers which is not a fully-human readable format; however, "string" data is stored as-is which makes reading it a reasonable task.
... View more
Labels:
04-05-2017
02:35 PM
Want to add some logs? A thread dump as the message instructs you to do? 🙂
... View more
04-04-2017
06:38 PM
You're probably running in the leftover tombstone from a delete: https://hbase.apache.org/book.html#_delete Compact your table and then run the import.
... View more
04-04-2017
05:46 PM
"What is the way to export to file system directory?"
You cannot do this. The MapReduce job can only write to HDFS as it is running across many nodes. Use the HDFS cli to copy the files to the local FS if you have this requirement.
"Will import work only if the table is empty of data?"
No, the import job does not require the table to be empty.
"Is it possible to append data with import ie table already has some data and we want to add more data with import."
This is essentially how the Import job works. The original timestamp on the exported data is preserved. So, if your destination table has a Key with a newer timestamp, you would not see the older data after import.
"Also is there any way to extract the table schemas including 'create
table' from hbase - or is 'describe <table>' the only way?"
Describe is the only way. However, you may want to consider using HBase Snapshots instead of these Export/Import mapreduce jobs. Snapshots implicitly hold onto the schema, but, upon restore, would re-set the table to the exact state (as opposed to Import's "merge")
... View more
04-04-2017
03:34 PM
Sounds like the output of a MapReduce job that you ran. HBase does not store anything outside of the value of "hbase.rootdir" in hbase-site.xml. On HDP, this defaults to "/apps/hbase/data". As long as you have no changed this configuration property, you can rest assured that HBase is not actively referring to files stored in "/tmp".
... View more