Adding Hive SerDe jar on SparkSQL Thrift Server

[#1]

I have Hive tables that point to JSON files as contents and these tables need JSON SerDe jar (from here) in order to query the tables. In Cloudera VM (Quickstart CDH 5.4.0), I can simply execute in Hive or Beeline CLI:

ADD JAR /<local-path>/json-serde-1.0.jar;

and then I am able to perform SELECT queries on my Hive tables.

I need to use these Hive tables as data sources for my Tableau (installed in Windows, my host machine), so I start the Thrift server in Spark.

For Hive tables that does not contain JSON (and does not require the SerDe), Tableau can connect and read the tables easily.

When it comes to the Hive tables that contain JSON data, however, it looks like Tableau cannot find the Hive JSON SerDe jar, and I get the following error:

'java.lang.RuntimeException: MetaException(message:java.lang.ClassNotFoundException Class org.openx.data.jsonserde.JsonSerDe not found)'.

How do I add the Hive JSON SerDe jar so that Tableau can read the Hive JSON tables?

[#2 · Accepted solution]

CDH5 Hive includes a JSON SerDe from the HCatalog component. Please use that instead:

Jar path: /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar
DDL line snippet: ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"

[#3]

Hi Harsh,

Is the library installed by default on the Hive process classpath.

CREATE TABLE jsontest(a string, b string)
ROW FORMAT SERDE "org.apache.hive.hcatalog.data.JsonSerDe"
STORED AS TEXTFILE;

=>

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.apache.hive.hcatalog.data.JsonSerDe

[#4]

So you must call add jar /opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar before executing the DDL. 

[#5]

I am currently using the following hive version

hive-1.1.0-cdh5.8.0, however when i tried searching the hive-hcatalog-core.jar, I could not find any directory named

/opt/cloudera/parcels/CDH/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core.jar. Any idea where can I download it from? need it to run a json data

[#6]

You can search it from the console using command

$ locate *hive-hcatalog-core*.jar