Created on 03-07-2017 12:22 AM - edited 09-16-2022 04:12 AM
Hi,
[ { "Classification": "core-site", "Properties": { "fs.s3a.awsAccessKeyId":"<aws key>", "fs.s3a.awsSecretAccessKey": "<aws secret key>" } }, { "Classification": "hive-site", "Properties": { "hive.metastore.warehouse.dir":"s3://<bucket_name>/<hive-folder>", "javax.jdo.option.ConnectionURL": "jdbc:mysql://<rds-url>:3306/hivedb?createDatabaseIfNotExist=true", "javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver", "javax.jdo.option.ConnectionUserName": "<db_user>", "javax.jdo.option.ConnectionPassword": "db_pass", "hive.exec.scratchdir":"/hive_temp/", "hive.exec.stagingdir" : "${hive.exec.scratchdir}/${user.name}/.staging", "hive.exec.dynamic.partition.mode":"nonstrict", "hive.exec.parallel":"true", "hive.exec.compress.intermediate":"true", "hive.optimize.index.filter":"true", "hive.optimize.index.groupby":"true", "hive.cluster.delegation.key.update-interval":"31536000000", "hive.cluster.delegation.token.renew-interval":"31536000000", "hive.cluster.delegation.token.max-lifetime":"31536000000" } }, { "Classification": "hue-ini", "Properties": {}, "Configurations": [ { "Classification": "desktop", "Properties": {"user_access_history_size":"50", "time_zone":"Europe/Berlin" }, "Configurations": [ { "Classification": "database", "Properties": { "name": "hue_db", "user": "hue_user", "password": "hue_pass", "host": "<rds_host>", "port": "3306", "engine": "mysql" }, "Configurations": [] } ] }, ## HUE AWS { "Classification": "aws", "Properties": {}, "Configurations": [ { "Classification": "aws_accounts", "Properties": {}, "Configurations": [ { "Classification": "default", "Properties": {"allow_environment_credentials": "False", "region": "eu-central-1"} } ] }] } ] } ]
Created 03-08-2017 08:27 AM
Created 03-08-2017 10:58 AM
I finally found the reason. It appears that launching the emr cluster with the following 2 hive configuration caused the hiveserver2 to fail on start.
"hive.exec.scratchdir":"/hive_temp/",
"hive.exec.stagingdir" : "${hive.exec.scratchdir}/${user.name}/.staging"
So after removing them, Hue can connect to the remove database and I have no problem run queries.
However I am still getting misconfiguration for cannot access the s3 path that I specified as my default dwh in hive-site:
"hive.metastore.warehouse.dir":"s3://<bucket_name>/<hive-folder>",
Any ideas how to fix it?
Thanks for your help