question Re: Adding Hive SerDe jar on SparkSQL Thrift Server in Archives of Support Questions (Read Only)

Adding Hive SerDe jar on SparkSQL Thrift Server

jdb — Fri, 16 Sep 2022 09:51:28 GMT

I have Hive tables that point to JSON files as contents and these tables need JSON SerDe jar (from here) in order to query the tables. In Cloudera VM (Quickstart CDH 5.4.0), I can simply execute in Hive or Beeline CLI:

ADD JAR /<local-path>/json-serde-1.0.jar;

and then I am able to perform SELECT queries on my Hive tables.

I need to use these Hive tables as data sources for my Tableau (installed in Windows, my host machine), so I start the Thrift server in Spark.

For Hive tables that does not contain JSON (and does not require the SerDe), Tableau can connect and read the tables easily.

When it comes to the Hive tables that contain JSON data, however, it looks like Tableau cannot find the Hive JSON SerDe jar, and I get the following error:

'java.lang.RuntimeException: MetaException(message:java.lang.ClassNotFoundException Class org.openx.data.jsonserde.JsonSerDe not found)'.

How do I add the Hive JSON SerDe jar so that Tableau can read the Hive JSON tables?

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

Harsh J — Sat, 19 Dec 2015 18:18:22 GMT

CDH5 Hive includes a JSON SerDe from the HCatalog component. Please use that instead:

Jar path: /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar
DDL line snippet: ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

WilliamQuby — Wed, 09 Mar 2016 16:13:42 GMT

Hi Harsh,

Is the library installed by default on the Hive process classpath.

CREATE TABLE jsontest(a string, b string)
ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"
STORED AS TEXTFILE;

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.apache.hive.hcatalog.data.JsonSerDe

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

WilliamQuby — Wed, 09 Mar 2016 16:18:58 GMT

So you must call add jar /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar before executing the DDL.

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

Abhijeet87 — Thu, 27 Apr 2017 03:22:46 GMT

I am currently using the following hive version

hive-1.1.0-cdh5.8.0, however when i tried searching the hive-hcatalog-core.jar, I could not find any directory named

/opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar. Any idea where can I download it from? need it to run a json data

Re: Adding Hive SerDe jar on SparkSQL Thrift Server

iraz098 — Fri, 20 Oct 2017 08:15:33 GMT

You can search it from the console using command

$ locate *hive-hcatalog-core*.jar