Created 02-22-2017 03:07 AM
Hi
As part of this tutorial on the Hortonworks site, I used a maven to compile the Json SerDe as instructed. I placed the Jar file into /usr/hdp/2.4.3.0-227/hive/lib. The tutorial says to place the same file in hive2/lib folder but there is no such folder in my HDP 2.4 box.
Interestingly, if I just do a select on the table that explicitly uses Serde, I get no error. The error only comes when Tez is doing a Join condition.
CREATE EXTERNAL TABLE IF NOT EXISTS tweets_text( tweet_id bigint, created_unixtime bigint, created_time string, lang string, displayname string, time_zone string, msg string) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' LOCATION '/tmp/tweets_staging';
No Errors. Results are returned successfully. The following query or any other join query, gives the error message below.
-- Compute sentiment create view IF NOT EXISTS l1 as select tweet_id, words from tweets_text lateral view explode(sentences(lower(msg))) dummy as words; create view IF NOT EXISTS l2 as select tweet_id, word from l1 lateral view explode( words ) dummy as word; create view IF NOT EXISTS l3 as select tweet_id, l2.word, case d.polarity when 'negative' then -1 when 'positive' then 1 else 0 end as polarity from l2 left outer join dictionary d on l2.word = d.word;
Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:265) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.openx.data.jsonserde.JsonSerDe not found at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:347) at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:382) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:227) ... 15 more Caused by: java.lang.ClassNotFoundException: Class
I would be most grateful for a quick resolution to this.
Thanks
Regards,
Adeel
Created 02-22-2017 03:27 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 02-22-2017 03:27 AM
Want to get a detailed solution you have to login/registered on the community
Register/LoginCreated 02-22-2017 04:17 AM
Hi Jay,
Sincere thanks for solving this problem for me. Have a great day!
Thanks
Adeel