question Re: Problem in inserting data into hbase table through hive in Archives of Support Questions (Read Only)

Problem in inserting data into hbase table through hive

Radius — Fri, 16 Sep 2022 08:46:38 GMT

Trying to insert data into hbase table through hive.:

Create table query:
CREATE TABLE hbase_testing(id int, name string, password string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "hbase_testing");

Insert Query:

INSERT OVERWRITE TABLE hbase_test2 SELECT * FROM user WHERE key=1;

Error:

java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja
java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.ja

Re: Problem in inserting data into hbase table through hive

dvohra — Tue, 13 Aug 2013 14:01:48 GMT

For Hive add the following JARs (version may be different) to classpath.

/opt/hive/lib/hive-hbase-handler-0.8.0-SNAPSHOT.jar
/opt/hive/lib/hbase-0.90.1-SNAPSHOT.jar
/opt/hive/lib/zookeeper-3.3.1.jar

Add to classpath either by adding to the Hadoop lib dir (/usr/lib/hadoop/lib) or with the add jar command.

add jar /opt/hive/lib/hive-hbase-handler-0.8.0-SNAPSHOT.jar
add jar /opt/hive/lib/hbase-0.90.1-SNAPSHOT.jar
add jar /opt/hive/lib/zookeeper-3.3.1.jar

Re: Problem in inserting data into hbase table through hive

Radius — Wed, 14 Aug 2013 05:43:02 GMT

Thnks but nothing happened...same error again.

Hey can you please mention steps of how to insert data into hbase table through hive. I think i am wrong somewhere.

Please help !!!!!

Re: Problem in inserting data into hbase table through hive

dvohra — Wed, 14 Aug 2013 14:03:31 GMT

bin/hive --auxpath /hadoop/projects/hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar,/hadoop/projects/hive-0.9.0/lib/hbase-0.92.0.jar,/hadoop/projects/hive-0.9.0/lib/zookeeper-3.3.4.jar,/hadoop/projects/hive-0.9.0/lib/guava-r09.jar -hiveconf hbase.master=localhost:60000

hive> CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz");

hive> INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM pokes WHERE foo=1;

Re: Problem in inserting data into hbase table through hive

Radius — Fri, 16 Aug 2013 05:55:31 GMT

I have followed same steps as you have mentioned. But still error came. Please verify it & please provide me some another solution.

bin/hive --auxpath /usr/lib/hive/lib/hive-hbase-handler-0.1 1.0.1.3.0.0-107.jar, /usr/lib/hive/lib/hbase-0.94.6.1.3.0.0-107-security.jar, /u sr/lib/hive/lib/zookeeper-3.4.5.1.3.0.0-107.jar, /usr/lib/hive/lib/guava-11.0.2. jar, -hiveconf hbase.master=localhost:60000

13/08/15 22:42:42 WARN conf.HiveConf: DEPRECATED: Configuration property hive.me tastore.local no longer has any effect. Make sure to provide a valid value for h ive.metastore.uris if you are connecting to a remote metastore.

Logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-commo n-0.11.0.1.3.0.0-107.jar!/hive-log4j.properties

hive> CREATE TABLE hbase_table_5(key int, value string) STORED BY 'org.apache.ha doop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mappin g" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xxx");

OK
Time taken: 4.99 seconds

hive> INSERT OVERWRITE TABLE hbase_table_5 SELECT * FROM user WHERE key=1;

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201307292222_0028, Tracking URL = http://cloudera:50030/jobdet ails.jsp?jobid=job_201307292222_0028
Kill Command = /usr/lib/hadoop/libexec/../bin/hadoop job -kill job_201307292222 _0028
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0
2013-08-15 22:44:08,760 Stage-0 map = 0%, reduce = 0%
2013-08-15 22:44:29,849 Stage-0 map = 100%, reduce = 100%
Ended Job = job_201307292222_0028 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://cloudera:50030/jobdetails.jsp?jobid=job_201307292222_002 8
Examining task ID: task_201307292222_0028_m_000002 (and more) from job job_20130 7292222_0028

Task with the most failures(4):
-----
Task ID:
task_201307292222_0028_m_000000

URL:
http://cloudera:50030/taskdetails.jsp?jobid=job_201307292222_0028&tipid=task_20 1307292222_0028_m_000000
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.jav a:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:6 4)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.ja va:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma tion.java:1232)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.jav a:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.jav a:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:6 4)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.ja va:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.jav a:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:1 21)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPoint erException
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSink Operator.java:385)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.j ava:407)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOper ator.java:62)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.j ava:407)
at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOper ator.java:78)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.j ava:407)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableSc anOperator.java:186)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.j ava:543)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:1 00)
... 22 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSink Operator.java:322)
... 38 more

FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRe dTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

Re: Problem in inserting data into hbase table through hive

dvohra — Fri, 16 Aug 2013 14:16:42 GMT

Some of the jars are not added in the hive --auxpath command.

Also add /hbase-0.94.1/hbase-0.94.6.jar and hive_contrib.jar. The version may be different.

Re: Problem in inserting data into hbase table through hive

Radius — Fri, 23 Aug 2013 05:56:26 GMT

Above problem has been solved by adding hbase.jar in /usr/lib/hadoop/lib & add some jar files like hive-hbase-handler.jar, hbase.jar, guava.jar, zookeeper.jar, hive-contrib.jar in auxpth in hive-site.xml..

Thanks

Surbhi