Member since
11-06-2016
5
Posts
2
Kudos Received
0
Solutions
03-31-2017
05:48 PM
I have a query trying to execute in hive context in spark using yarn cluster Version Spark 1.5.2 Time taking for exact same query Hive in TEZ : 3 min Hive context.sql in SPARK: 14 min Findings: spark execution is done in 52 stages which completed in ~4min but Insert overwrite into partition is taking ~8-9 min (where data is copying from hive staging to hdfs) I have seen problem is raised by many people, but I can't find any answer. This is very critical. Note: Please suggest the optimal way to execute in less time Please answer in detail so it will be helpful for others @Benjamin Leonhardi, @Ravi Mutyala , @gopal
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Spark
01-18-2017
05:06 PM
hive_client = HiveClient(db_host='XXXXXXXXX.scglobaluat.aduat.scotiacapital.com', port=10000, authMechanism='GSSAPI', user='XXXXXXX', password='XXXXXX',database='tsz', kbservice='hive')
The below is error code: Please let me know where is the issue
Traceback (most recent call last): File "C:/Users/speesary/connct_server", line 33, in <module>
database='tsz', kbservice='hive')
File "C:/Users/speesary/connct_server", line 17, in __init__
kerberos_service_name=kbservice File "C:\Users\speesary\AppData\Roaming\Python\Python27\site-packages\impala\dbapi.py", line 147, in connect
auth_mechanism=auth_mechanism)
File "C:\Users\speesary\AppData\Roaming\Python\Python27\site-packages\impala\hiveserver2.py", line 658, in connect
transport.open() File "C:\Users\speesary\AppData\Roaming\Python\Python27\site-packages\thrift_sasl\__init__.py", line 72, in open
message=("Could not start SASL: %s" % self.sasl.getError()))
thrift.transport.TTransport.TTransportException: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: Unable to find a callback: 2
Process finished with exit code 1
... View more
11-07-2016
04:24 AM
@Dale Bradman can you please help me to solve this issue. it is much appreciated
... View more
11-07-2016
04:24 AM
@Artem Ervits can you please help me to solve this issue. it is much appreciated
... View more
11-07-2016
04:24 AM
2 Kudos
conn_config ={'krb_host':'hostname','krb_service':'hive'}
pyhs2.connect(host='hostname',
port=10000,
authMechanism="KERBEROS",
password="********",
user='hostname@XXXXXXXXXXXXXXXXXXXX') Error Encountered:
user='XXXXXXX@XXXXXXXXXXXXXXXXXXXX')
File "build\bdist.win-amd64\egg\pyhs2\__init__.py", line 7, in connect
File "build\bdist.win-amd64\egg\pyhs2\connections.py", line 46, in __init__
File "build\bdist.win-amd64\egg\pyhs2\cloudera\thrift_sasl.py", line 66, in open
thrift.transport.TTransport.TTransportException: Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: Unable to find a callback: 2
checked below string but still have same problem
http://stackoverflow.com/questions/29814207/python-connect-to-hive-use-pyhs2-and-kerberos-authentication
Can please some one help to solve ths problem. Your help is much appreciated
... View more
Labels:
- Labels:
-
Apache Hive