Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Access a partitioned table created in Impala using Hive

Solved Go to solution
Highlighted

Access a partitioned table created in Impala using Hive

Contributor

I've recently created the following table in Impala....

create table hpchistperf.oozie_job_stats
(
name string,
start_time timestamp,
end_time timestamp,
rows_inserted integer,
last_id_loaded integer,
primary key(name,start_time))
PARTITION BY HASH PARTITIONS 16
stored as kudu;

 

I read that there was no problem creating the table in Impala and it could be accessed in Hive since they share the same metadata.

 

The problem is that when I run the query on Hive it gives me the following error:

FAILED: RuntimeException java.lang.ClassNotFoundException: org.apache.kudu.mapreduce.KuduTableInputFormat

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Access a partitioned table created in Impala using Hive

Contributor

@Hari 

 

When you create a table in Impala with data in kudu, the metadata resides in the Hive metastore and you can see the table in hive as well as both Hive and Impala share the same metadata. But if you try to access the data by running a query in hive over kudu table, you are bound to get the below error.

 

FAILED: RuntimeException java.lang.ClassNotFoundException: org.apache.kudu.mapreduce.KuduTableInputFormat

 

This is because in CDH, Hive doesn't support accessing Kudu. It is added in Hive 4.0 in HIVE-12971 and is designed to work with Kudu 1.2+

 

You can try out in CDP where in this feature has been added.

 

https://cwiki.apache.org/confluence/display/Hive/Kudu+Integration

https://issues.apache.org/jira/browse/HIVE-12971

View solution in original post

1 REPLY 1
Highlighted

Re: Access a partitioned table created in Impala using Hive

Contributor

@Hari 

 

When you create a table in Impala with data in kudu, the metadata resides in the Hive metastore and you can see the table in hive as well as both Hive and Impala share the same metadata. But if you try to access the data by running a query in hive over kudu table, you are bound to get the below error.

 

FAILED: RuntimeException java.lang.ClassNotFoundException: org.apache.kudu.mapreduce.KuduTableInputFormat

 

This is because in CDH, Hive doesn't support accessing Kudu. It is added in Hive 4.0 in HIVE-12971 and is designed to work with Kudu 1.2+

 

You can try out in CDP where in this feature has been added.

 

https://cwiki.apache.org/confluence/display/Hive/Kudu+Integration

https://issues.apache.org/jira/browse/HIVE-12971

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here