Unable to import data into hbase table through Importtsv.

[#1]

Hi, I am using Apache Hbase (Version 1.1.3).

Sample data used:

simple structure that I have got for an industrial sensor

 id, temp:in,temp:out,vibration,pressure:in,pressure:out
 5842,50,30,4,240,340

executed the Loadtsv statement as following

hbase> hbase org.apache.hadoop.hbase.mapreduce.ImportTsv-Dimporttsv.separator=,-Dimporttsv.columns="HBASE_ROW_KEY,id,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor /user/hbase.csv

import the same data (header removed) - got this error - syntax error, unexpected ','

then removed the 'ID' column in '-Dimporttsv.columns'

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator= ',' -Dimporttsv.columns="HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out" sensor /user/hbase.csv

getting

syntax error, unexpected tIDENTIFIER

Please help

Thanks, Sridharan

[#2]

I don't think it makes a difference but can you try and put a space between the separator=, and -Dimporttsv

[#3]

-Dimporttsv.separator= ','

There was extraneous space between equals and ','

Please remove it

[#4]

Hi Sunil and Ted, Thanks for your reply,

I tried both your suggestions, but no luck

hbase(main):008:0> hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out' sensor user/hbase.csv

Getting

SyntaxError: (hbase):8: syntax error, unexpected tIDENTIFIER

The table structure is

hbase(main):009:0> describe 'sensor' Table sensor is ENABLED sensor COLUMN FAMILIES DESCRIPTION {NAME => 'pressure', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_V ERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'} {NAME => 'temp', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSI ONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'} {NAME => 'vibration', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_ VERSIONS => '0', TTL => 'FOREVER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'true'}

Should anything be modified here? How to do that please?

Thanks,

Sridharan

[#5]

The example from http://hbase.apache.org/book.html#importtsv doesn't use quote around columns parameter.

Can you pastebin the output from ImportTsv so that we can see the full error trace ?

Please turn on debug logging.

[#6]

I created a table with schema similar to yours.

create 't2', {NAME => 'pressure'}, {NAME => 'temp'}, {NAME => 'vibration'}

I created hbase.csv with the line in your first post.

I then executed the following (HDP 2.5.0.0-267 which is 1.1.2 with some patches):

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out' t2 hbase.csv

2016-06-17 17:23:18,142 INFO [main] mapreduce.Job: Running job: job_1462297012839_0003

2016-06-17 17:23:29,332 INFO [main] mapreduce.Job: Job job_1462297012839_0003 running in uber mode : false 2016-06-17 17:23:29,335 INFO [main] mapreduce.Job: map 0% reduce 0%

2016-06-17 17:23:38,466 INFO [main] mapreduce.Job: map 100% reduce 0%

2016-06-17 17:23:38,478 INFO [main] mapreduce.Job: Job job_1462297012839_0003 completed successfully 2016-06-17 17:23:38,670 INFO [main] mapreduce.Job: Counters: 31

[#7]

hbase(main):001:0> scan 't2'

ROW COLUMN+CELL

5842 column=pressure:in, timestamp=1466184191234, value=240

5842 column=pressure:out, timestamp=1466184191234, value=340

5842 column=temp:in, timestamp=1466184191234, value=50

5842 column=temp:out, timestamp=1466184191234, value=30

5842 column=vibration:, timestamp=1466184191234, value=4

1 row(s) in 0.4840 seconds

[#8]

Hi Ted, Thanks again for your help. I tried the same steps as you did again. But still no luck.

I have a basic question.

'hbase.csv' is placed in hdfs in your case? I placed it in HDFS folder. Should that be placed in the hbase directory?

Thanks,

Sridharan

[#9]

Getting the below message as output

hbase(main):003:0> hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out' t2 user/hbase.csv SyntaxError: (hbase):3: syntax error, unexpected tIDENTIFIER hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out' t2 user/hbase.csv ^

[#10]

Are you running the command from HBase shell? You don't need to run HBase shell but just run the command from Unix (or Windows) shell.

> /usr/bin/hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,temp:in,temp:out,vibration,pressure:in,pressure:out' t2 user/hbase.csv