Error when copying table from Hive to HBase.

[#1]

Hello. I am trying to copy an existing table from Hive to a new table in HBase with the following query:

--------------------------------------------------

CREATE TABLE hbase_lineitem (part_key int, item_id int, category_id int, dept_id int, quantity int, price double, discount double, tax double, flag_1 string, flag_2 string)

STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'

WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":part_key,lineitem:item_id,lineitem:category_id,lineitem:dept_id,lineitem:quantity,lineitem:price,lineitem:discount,lineitem:tax,lineitem:flag_1,lineitem:flag_2");

INSERT OVERWRITE TABLE hbase_lineitem SELECT * FROM lineitem;

---------------------------------------------------

However, I am receiving the following error:

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException org.apache.hadoop.hive.hbase.HBaseSerDe: columns has 10 elements while hbase.columns.mapping has 11 elements (counting the key if implicit))

I only count 10 elements in both of my lists, so can anyone help me figure out what mistake I am making? Thanks.

[#2 · Accepted solution]

I would guess that the problem is that you are not explicitly providing a :key column which makes your value actually have 10+1 entries instead of just 10.

You will want to map one of the Hive columns you are selecting into the HBase rowKey. This is done by including a ":key" entry in the value for "hbase.columns.mapping". This is what defines uniqueness in each record.

See https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-ColumnMapping for more information.

[#3]

This fixed the issue, appreciate it!

[#4]

If I correctly understand, you need to change :part_key to :key.