i'm working in CDH5.13 with Hadoop 2.6.0-cdh5.13.0 , mongodb v4.2.5 and Hive 1.1.0-cdh5.13.0.
I downloaded the mongo-hadoop project here:https://github.com/mongodb/mongo-hadoop/archive/refs/tags/r1.5.1.tar.gz
I built the mongo-hadoop-core-1.5.1.jar and the mong-hadoop-hive-1.5.1jar using the ./gradlew jar command. I also downloaded the mongo-java-driver-3.12.7.jar on this site :https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver/3.12.7
As my table is already in mongoDB, i created the hive script below:
ADD JAR /home/cloudera/Downloads/mongo-java-driver- 3.12 . 7 .jar;
ADD JAR /home/cloudera/lib/mongo-hadoop-core- 1.5 . 1 .jar;
ADD JAR /home/cloudera/lib/mongo-hadoop-hive- 1.5 . 1 .jar;
CREATE TABLE DataEntreprise
(
id STRING,
Entreprise INT,
CA3 INT, Benef3 INT, Perte3 INT,
Renta2020 INT,
Renta2019 INT,
Renta2018 INT,
AgeAct1 INT, PartAct1 INT,
AgeAct2 INT, PartAct2 INT,
AgeAct3 INT, PartAct3 INT,
AgeAct4 INT, PartAct4 INT,
AgeAct5 INT, PartAct5 INT,
AgeAct6 INT, PartAct6 INT
)
STORED BY 'com .mongodb.hadoop.hive.MongoStorageHandler'
WITH SERDEPROPERTIES( 'mongo .columns.mapping '=' { "id" : "_id" , "CA3" : "Chiffre Affaire - 3 ans (K€)" , "Benef3" : "Bénéfice - 3ans (K€)" , "Perte3" : "Perte - 3ans (K€)" , "Renta2020" : "Rentabilité 2020 (%)" , "Renta2019" : "Rentabilité 2019 (%)" , "Renta2018" : "Rentabilité 2018 (%)" , "AgeAct1" : "Age Actionnaire 1 (ans)" , "PartAct1" : "Part Actionnaire 1 (%)" , "AgeAct2" : "Age Actionnaire 2 (ans)" , "PartAct2" : "Part Actionnaire 2 (%)" , "AgeAct3" : "Age Actionnaire 3 (ans)" , "PartAct3" : "Part Actionnaire 3 (%)" , "AgeAct4" : "Age Actionnaire 4 (ans)" , "PartAct4" : "Part Actionnaire 4 (%)" , "AgeAct5" : "Age Actionnaire 5 (ans)" , "PartAct5" : "Part Actionnaire 5 (%)" , "AgeAct6" : "Age Actionnaire 6 (ans)" , "PartAct6" : "Part Actionnaire 6 (%)" }')
TBLPROPERTIES( 'mongo .uri '=' mongodb: //localhost:27017/Data.Test');
After executing the previous script, the table is created in Hive but without any data or columns; i have these two erros:
Error while processing statement: FAILED: Execution Error , return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.ClassNotFoundException Class com.mongodb.hadoop.hive.BSONSerDe not found
Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Error in loading storage handler.com.mongodb.hadoop.hive.MongoStorageHandler
Please HELP!!
... View more
