I am trying to import data to hbase using importtsv. I tried couple of examples from online to learn and they worked fine.
Fx with below command I get the success and see the table on hbase shell.
bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns=HBASE_ROW_KEY,cf tab4 /user/hduser/gutenberg/simple1.txt
2016-06-20 17:08:03,138 INFO [main] mapreduce.Job: map 100% reduce 0%
2016-06-20 17:08:03,139 INFO [main] mapreduce.Job: Job job_local1517188704_0001 completed successfully
2016-06-20 17:08:03,269 INFO [main] mapreduce.Job: Counters: 24
File System Counters
FILE: Number of bytes read=4018816
FILE: Number of bytes written=25538098but the actual data I want to upload has lots of columns and rows. (237 to 400k respectively) To check out integrity I uploaded and tried to import a 237 column 10 row version of the data, to make it visible I am pasting the part of the data and the command.
1,date,serial,serial,date,bsrid,SW version,bsr Group Id,Processed kpi Number,reserved....
2,20151206,1211080003,1211080003,20151206,103,30092,0,24,0,0,...
Here is a part of the command I am running.
bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns=HBASE_ROW_KEY,date,serial,serial_1,date_1,bsrid,SW_version,bsr_Group_Id,Processed_kpi_Number, (until 237th column) tab5 /user/hduser/KPI-Dumps/test.csv
When I try to run the command it gets the below status and when I type jps command I can't see the HRegionServer anymore and I have to restart HBase
2016-06-20 16:20:07,804 INFO [communication thread] mapred.LocalJobRunner: map
2016-06-20 16:20:08,581 INFO [main] mapreduce.Job: map 100% reduce 0%
2016-06-20 16:21:15,159 INFO [hconnection-0x1abc383-metaLookup-shared--pool5-t2] client.RpcRetryingCaller: Call exception, tries=10, retries=35, started=68390 ms ago, cancelled=false, msg=row 'tab5,1,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=cellops-optiplex-gx620,16201,1466432370372, seqNum=0
2016-06-20 16:21:35,227 INFO [hconnection-0x1abc383-metaLookup-shared--pool5-t2] client.RpcRetryingCaller: Call exception, tries=11, retries=35, started=88487 ms ago, cancelled=false, msg=row 'tab5,1,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=cellops-optiplex-gx620,16201,1466432370372, seqNum=0
There is no typo or any missing comma or etc, double checked. My only guess is the difference of the table related to the column order is different, Hbase created the columns based on alphabetical order instead of the version I wanted it to create even though I gave the correct order based on the file I want to upload. Do you think it gets confused during upload?
From the created table
..
{NAME => 'Vs_SuccActiveCallReDirectUMTS'}
{NAME => 'bsr_Group_Id'}
{NAME => 'bsrid'}
{NAME => 'date'}
{NAME => 'date_1'}
{NAME => 'reserved'}
{NAME => 'reserved_1'}
{NAME => 'reserved_10'}
{NAME => 'reserved_11'}
...
hbase-site.xml is like this and its a pseudo distributed installation with zookeper also installed manually.
<configuration>
<property>
<name>hbase.rootdir</name>
<value>/usr/local/hadoop/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/usr/local/Hbase/zookeeper</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>zookeeper.znode.parent</name>
<value>/hbase-unsecure</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>hbase.rpc.timeout</name>
<value>120000</value>
</property>
<property>
<name>hbase.client.scanner.timeout.period</name>
<value>120000</value>
</property>
<property>
<name>hbase.cells.scanned.per.heartbeat.check</name>
<value>10000</value>
</property>
</configuration>
hduser@cellops-OptiPlex-GX620:/usr/local/Hbase$ jps
9712 NodeManager
31301 QuorumPeerMain
18294 HMaster
9481 ResourceManager
9289 SecondaryNameNode
18414 HRegionServer
9023 DataNode
8799 NameNode
18847 Jps
