Support Questions

Find answers, ask questions, and share your expertise

Where to put json-serde-1.3.7-jar-with-dependencies jar file

avatar

First I add the hive-serdex-1.0-snapshot jar file

hive> ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar;
Added [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar] to class path
Added resources: [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar]

And than I add the json-serde-1.3.7 jar file (I believe this is why I get the error)

 

hive> ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar;
Added [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar] to class path
Added resources: [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar]

 

But when I try to create this table in hive:

CREATE EXTERNAL TABLE tweets (    
    id BIGINT,
    created_at STRING,
    source STRING,
    favorited BOOLEAN,
    retweet_count INT,
    retweeted_status STRUCT<
    text:STRING,
    `user`:STRUCT<screen_name:STRING,name:STRING>>,
    entities STRUCT<
    urls:ARRAY<STRUCT<expanded_url:STRING>>,
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>,
    hashtags:ARRAY<STRUCT<text:STRING>>>,
    text STRING,
    `user` STRUCT<
    screen_name:STRING,
    name:STRING,
    friends_count:INT,
    followers_count:INT,
    statuses_count:INT,
    verified:BOOLEAN,
    utc_offset:INT,
    time_zone:STRING>,
    in_reply_to_screen_name STRING
)
ROW FORMAT SERDE 'com.cloudera.hive.SERDE.JSONSerDe'
LOCATION '/user/cloudera/flume/tweets'

I get this error:

Error while processing statement: FAILED: Execution Error, return code 1 
from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: 
com.cloudera.hive.SERDE.JSONSerDe

I believe the error is because I put the json-serde-1.3.7-jar-with-dependencies jar file in the wrong location. Where should it go?

 

This is where I download the json-serde-1.3.7 jar file

download json-serde-file for cdh5

1 REPLY 1

avatar
Super Guru

Hi,

 

The JSON class in json-serde-1.3.7-jar-with-dependencies.jar is org.openx.data.jsonserde.JsonSerDe, not com.cloudera.hive.SERDE.JSONSerDe. So did you mean to use com.cloudera.hive.SERDE.JSONSerDe or org.openx.data.jsonserde.JsonSerDe?

 

Can you give me the output of :

 

jar tf /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar

 

so that I can confirm the classes in this jar?

 

Thanks