Member since
12-30-2015
164
Posts
29
Kudos Received
10
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
18610 | 01-07-2019 06:17 AM | |
937 | 12-27-2018 07:28 AM | |
3368 | 11-26-2018 10:12 AM | |
1003 | 11-16-2018 12:15 PM | |
3049 | 10-22-2018 09:31 AM |
12-31-2018
09:16 AM
Hi Vinay, does the spark user is having sufficient privileges to access hive db and table?
... View more
12-27-2018
02:32 PM
Hi @Shantanu
Sharma, Seems issue with the metadata directory for hbase in zookeeper. if it's a fresh installation please remove the hbase metadata directory by using zkcli.sh and reinstall the hbase again. Thank you.
... View more
12-27-2018
02:28 PM
Hi @Vinay please use the below syntax to connect hive from spark: spark-shell --conf spark.sql.hive.hiveserver2.jdbc.url="jdbc:hive2://************************:10000/" spark.datasource.hive.warehouse.load.staging.dir="/tmp" spark.hadoop.hive.zookeeper.quorum="ip.************:2181" --jars /usr/hdp/current/hive_warehouse_connector/hive-warehouse-connector-assembly-1.0.0.3.0.0.0-1634.jar Refer the below document for database operations by using spark in hive: https://github.com/hortonworks-spark/spark-llap/tree/master Please accept the answer if it help's Thank you.
... View more
12-27-2018
10:51 AM
Hi @Vinay, Please connect hiveserver2 instead of hiveserver2Interactive by using below syntax: spark.sql.hive.hiveserver2.jdbc.url=jdbc:hive2://sidchadoop04.test.com:10000
... View more
12-27-2018
07:59 AM
Hi @Ilia K, You need to create the new config group under Yarn config and add the new nodemanagers into that group. Then override the yarn nodemanger properties for cpu and memory.
... View more
12-27-2018
07:28 AM
1 Kudo
Hi @Michael Bronson, To Clean the Zookeeper data directory follow the below support article https://community.hortonworks.com/content/supportkb/48795/how-to-purge-old-zookeeper-directory-files.html
... View more
12-27-2018
07:21 AM
Hi @Saurav Ranjit, Can you please check hive queries are running for other tables/schemas ? if other queries are running fine and particularly above mentioned query only failing you need to tune above mentioned query if all queries are failing with same vertex error , you need to tune hive and tez perimeters
... View more
12-12-2018
10:43 AM
Hello Team,
we had recently enabled https for azure storage and changed fs.defaultFS parameter in core-site.xml. we are able to access the hdfs file system . Now our hive queries are failing with the below error ?
Error: java.io.IOException: org.apache.hadoop.fs.azure.AzureException: com.microsoft.azure.storage.StorageException: The account being accessed does not support http. (state=,code=0)
we have changed the parameter for hive.aux.jars.path to use wasbs. still we are facing the same issue ?
do we missed anything in hive-site.xml to reach wasbs ?
Any help to overcome this issue much appreciated
Thanks in advance!
... View more
Labels:
- Labels:
-
Apache Hive
11-26-2018
10:12 AM
This problem occurs while the ticket is expired. resolved after generating the ticket by using kinit -kt user.keytab use@realm
... View more
11-26-2018
09:47 AM
[hdfs@ip-10-0-10-206 ~]$ hdfs dfs -ls /user
18/11/26 09:26:10 WARN ipc.Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
ls: DestHost:destPort ip-10-0-10-87:8020 , LocalHost:localPort ip-10-0-10-206/10.0.10.206:0. Failed on local exception: java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
... View more
Labels: