Hi As part of this tutorial on the Hortonworks site, I used a maven to compile the Json SerDe as instructed. I placed the Jar file into /usr/hdp/2.4.3.0-227/hive/lib. The tutorial says to place the same file in hive2/lib folder but there is no such folder in my HDP 2.4 box. Interestingly, if I just do a select on the table that explicitly uses Serde, I get no error. The error only comes when Tez is doing a Join condition. CREATE EXTERNAL TABLE IF NOT EXISTS tweets_text(
tweet_id bigint,
created_unixtime bigint,
created_time string,
lang string,
displayname string,
time_zone string,
msg string)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
LOCATION '/tmp/tweets_staging'; No Errors. Results are returned successfully. The following query or any other join query, gives the error message below. -- Compute sentiment
create view IF NOT EXISTS l1 as select tweet_id, words from tweets_text lateral view explode(sentences(lower(msg))) dummy as words;
create view IF NOT EXISTS l2 as select tweet_id, word from l1 lateral view explode( words ) dummy as word;
create view IF NOT EXISTS l3 as select
tweet_id,
l2.word,
case d.polarity
when 'negative' then -1
when 'positive' then 1
else 0 end as polarity
from l2 left outer join dictionary d on l2.word = d.word; Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:265) at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:149) ... 14 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.openx.data.jsonserde.JsonSerDe not found at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:347) at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:382) at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:227) ... 15 more Caused by: java.lang.ClassNotFoundException: Class I would be most grateful for a quick resolution to this. Thanks Regards, Adeel
... View more
