Created 07-06-2021 04:39 AM
Hi Team,
I am trying to connect the hive table from CDSW data science workbench using python. i am able to connect the hive from python using impala, when i try to execute the query i am getting below error,
SemanticException [Error 10265]: This command is not allowed on an ACID table default.atlastest with a non-ACID transaction manager. Failed command: SELECT * FROM `default`.`table_name`'), operationHandle=None)
PFB code ref:
import os
import pandas
from impala.dbapi import connect
from impala.util import as_pandas
# Specify HIVE_HS2_HOST host name as an environment variable in your project settings
HIVE_HS2_HOST='1.1.1.1'
# This connection string depends on your cluster setup and authentication mechanism
conn = connect(host=HIVE_HS2_HOST,
port=10000,
user = 'username',
password = 'password',
auth_mechanism='PLAIN',
kerberos_service_name='hive')
cursor = conn.cursor()
res = cursor.execute('SELECT * FROM `default`.`table_name`')
Kindly help me to close this issue
Created 07-11-2021 05:18 AM
@sur I believe you have to run the queries over Impala not Hive. I have tested below and it works several time in the past.
# Install Libraries
!pip3 install impyla==0.13.8
!pip3 install thrift_sasl==0.2.1
!pip3 install thrift==0.9.3
!pip3 install sasl==0.2.1
# Connect to Impala
from impala.dbapi import connect
conn = connect(host='host',
port=21050,
auth_mechanism='GSSAPI',
use_ssl=True,
kerberos_service_name='impala')
# Execute Query
sql = "select * from table"
cursor = conn.cursor()
cursor.execute(sql)
results = cursor.fetchall()
You can refer the below doc: https://docs.cloudera.com/cdsw/1.9.1/import-data/topics/cdsw-running-queries-on-impala-tables.html
Created on 07-11-2021 10:16 PM - edited 07-11-2021 10:36 PM
@GangWar Thanks for your reply; Actually i am trying to query on Hive table; and I have followed the these steps; https://docs.cloudera.com/cdsw/1.9.1/import-data/topics/cdsw-accessing-data-from-apache-hive.html
I am able to establish the connection and executed 'show tables;' -- it works fine as expected. The real problem comes only when i try to run select query on hive table. It is showing the above error which i mentioned early.
Created 07-12-2021 07:52 PM
It works on impala shell or hive beeline?
Created 07-12-2021 09:13 PM
@sur I believe you have to Set below properties before executing create table statement.
SET hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
SET hive.support.concurrency=true;
This seems a very known behaviour and I did a simple Googling and found the solutions on several places. Below are the references.
Hope this helps.