question Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable in Archives of Support Questions (Read Only)

Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

hadoopsmi — Tue, 22 Mar 2016 14:28:48 GMT

Jar -> mongo-hadoop-core-1.4.0,mongo-hadoop-hive-1.4.0,mongo-java-driver-2.10.1

hive> CREATE EXTERNAL TABLE minute_bars > ( > > id STRING, > Symbol STRING, > `Timestamp` STRING, > Day INT, > Open DOUBLE, > High DOUBLE, > Low DOUBLE, > Close DOUBLE, > Volume INT > ) > STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler' > WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id", > "Symbol":"Symbol", "Timestamp":"Timestamp", "Day":"Day", "Open":"Open", "High":"High", "Low":"Low", "Close":"Close", "Volume":"Volume"}') > TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/marketdata.minbars'); FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable hive>

Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

jyadav — Tue, 22 Mar 2016 15:15:36 GMT

You could try adding both the "mongo-hadoop-hive.jar" and "mongo-hadoop-core.jar" to the hive.aux.jars.path setting in your hive-site.xml.

You can simply add those jars in your hive shell like,

hive> add "somepath/mongo-hadoop-hive.jar"

hive> add "somepath/mongo-hadoop-core.jar"

Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

hadoopsmi — Tue, 22 Mar 2016 17:56:55 GMT

i alrady done that step

hive> add "somepath/mongo-hadoop-hive.jar"

hive> add "somepath/mongo-hadoop-core.jar"

Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

jyadav — Tue, 22 Mar 2016 22:31:57 GMT

Ok, can you also try using hive aux path?

Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

aervits — Wed, 23 Mar 2016 15:01:47 GMT

have you seen this https://community.hortonworks.com/content/repo/4538/hdp-mongo-tutorial.html ? Also mongo driver is now at 1.4.2, it may be worth your time to use latest jar. https://github.com/mongodb/mongo-hadoop/releases

Re: Mongodb with hive : Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. com/mongodb/hadoop/io/BSONWritable

aervits — Thu, 31 Mar 2016 06:40:36 GMT

I got it to work with the following in my repo I linked earlier

hdfs dfs -put drivers/* /tmp/udfs

beeline
!connect jdbc:hive2://localhost:10000 “” ””

add jar hdfs://sandbox.hortonworks.com:8020/tmp/udfs/mongo-hadoop-hive-1.5.0-SNAPSHOT.jar;
add jar hdfs://sandbox.hortonworks.com:8020/tmp/udfs/mongo-hadoop-core-1.5.0-SNAPSHOT.jar;
add jar hdfs://sandbox.hortonworks.com:8020/tmp/udfs/mongodb-driver-3.0.4.jar;
DROP TABLE IF EXISTS bars;
CREATE EXTERNAL TABLE bars
(
objectid STRING,
    Symbol STRING,
    TS STRING,
    Day INT,
    Open DOUBLE,
    High DOUBLE,
    Low DOUBLE,
    Close DOUBLE,
    Volume INT
)
STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES('mongo.columns.mapping'='{"objectid":"_id",
 "Symbol":"Symbol", "TS":"Timestamp", "Day":"Day", "Open":"Open", "High":"High", "Low":"Low", "Close":"Close", "Volume":"Volume"}')
TBLPROPERTIES('mongo.uri'='mongodb://localhost:27017/marketdata.minibars');