Reply
Highlighted
jdb
Explorer
Posts: 10
Registered: ‎07-19-2015
Accepted Solution

Adding Hive SerDe jar on SparkSQL Thrift Server

I have Hive tables that point to JSON files as contents and these tables need JSON SerDe jar (from here) in order to query the tables. In Cloudera VM (Quickstart CDH 5.4.0), I can simply execute in Hive or Beeline CLI:

 

ADD JAR /<local-path>/json-serde-1.0.jar;

and then I am able to perform SELECT queries on my Hive tables.

 

I need to use these Hive tables as data sources for my Tableau (installed in Windows, my host machine), so I start the Thrift server in Spark.

 

For Hive tables that does not contain JSON (and does not require the SerDe), Tableau can connect and read the tables easily.

 

When it comes to the Hive tables that contain JSON data, however, it looks like Tableau cannot find the Hive JSON SerDe jar, and I get the following error:

 

'java.lang.RuntimeException: MetaException(message:java.lang.ClassNotFoundException Class org.openx.data.jsonserde.JsonSerDe not found)'.

How do I add the Hive JSON SerDe jar so that Tableau can read the Hive JSON tables?

Posts: 1,565
Kudos: 287
Solutions: 239
Registered: ‎07-31-2013

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

CDH5 Hive includes a JSON SerDe from the HCatalog component. Please use that instead:

Jar path: /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar
DDL line snippet: ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"
Backline Customer Operations Engineer
New Contributor
Posts: 2
Registered: ‎03-09-2016

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

Hi Harsh,

 

Is the library installed by default on the Hive process classpath.

 

CREATE TABLE jsontest(a string, b string)
ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"
STORED AS TEXTFILE;

 

=>

 

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.apache.hive.hcatalog.data.JsonSerDe

 

 

New Contributor
Posts: 2
Registered: ‎03-09-2016

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

So you must call add jar /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar before executing the DDL. 

New Contributor
Posts: 1
Registered: ‎04-26-2017

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

I am currently using the following hive version

hive-1.1.0-cdh5.8.0, however when i tried searching the hive-hcatalog-core.jar, I could not find any directory named

/opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar. Any idea where can I download it from? need it to run a json data

Explorer
Posts: 7
Registered: ‎10-04-2016

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

You can search it from the console using command

$ locate *hive-hcatalog-core*.jar

Announcements