Cloudera Community
George_Cancro
user rank
Explorer

Re: pyHive interface does not seem to work for mor...

by Explorer in Support Questions
‎05-08-2017 05:31 PM
1 Kudo
‎05-08-2017 05:31 PM
1 Kudo
Did something that worked for me... from pyhive import hive conn = hive.Connection(host='<myhost>', port='<myport>', database='spin1', username='<a valid user>') # IMPORTANT** cursor = conn.cursor() print cursor.fetchall() IMPORTANT** - Before I was not using a valid username, I used 'hive' because of some earlier example I found....and eventhough simple queries worked...the more complex queries failed. Think this may have to do with write permissions...but don't know. Reason why I say this is that When I tried the same thing using beeline it threw a similar but different error that included something about write permissions being denied. when I switched to a valid username, the query worked in beeline. Probably a stupid mistake on my part...but wanted to write it up in case someone else repeats it. ... View more

pyHive interface does not seem to work for more di...

by Explorer in Support Questions
‎05-08-2017 04:39 PM
2 Kudos
‎05-08-2017 04:39 PM
2 Kudos
I followed the article on https://community.hortonworks.com/articles/97062/query-hive-using-python.html BUT...this only seems to work for small queries "show databases", "show tables", "SELECT * FROM X",....when I tried a JOIN like... 'SELECT * from spin1.patient JOIN cohortdefinitions.cohort_first25_spin1db ON patient.id=cohort_first25_spin1db.id' It fails. with >>>cursor.execute( 'SELECT * from spin1.patient JOIN cohortdefinitions.cohort_first25_spin1db ON patient.id=cohort_first25_spin1db.id') Traceback (most recent call last): File "<ipython-input-22-32a836d67b2f>", line 1, in <module> cursor.execute(connInfo['query']) File "/home/ace4user/anaconda/lib/python2.7/site-packages/pyhive/hive.py", line 240, in execute _check_status(response) File "/home/ace4user/anaconda/lib/python2.7/site-packages/pyhive/hive.py", line 362, in _check_status raise OperationalError(response) OperationalError: TExecuteStatementResp(status=TStatus(errorCode=1, errorMessage='Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask', sqlState='08S01', infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask:28:27', 'org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:335', 'org.apache.hive.service.cli.operation.SQLOperation:runQuery:SQLOperation.java:199', 'org.apache.hive.service.cli.operation.SQLOperation:runInternal:SQLOperation.java:228', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:276', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:468', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatement:HiveSessionImpl.java:444', 'sun.reflect.GeneratedMethodAccessor211:invoke::-1', 'sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43', 'java.lang.reflect.Method:invoke:Method.java:498', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78', 'org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36', 'org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63', 'java.security.AccessController:doPrivileged:AccessController.java:-2', 'javax.security.auth.Subject:doAs:Subject.java:422', 'org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1724', 'org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59', 'com.sun.proxy.$Proxy25:executeStatement::-1', 'org.apache.hive.service.cli.CLIService:executeStatement:CLIService.java:273', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:506', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1317', 'org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1302', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617', 'java.lang.Thread:run:Thread.java:745'], statusCode=3), operationHandle=None) ANY IDEAS???? ... View more
Labels:

Re: Query Hive Using Python

by Explorer in Community Articles
‎05-08-2017 04:30 PM
‎05-08-2017 04:30 PM
Great article...BUT...this only seems to work for small queries "show databases", "show tables", "SELECT * FROM X",....when I tried a JOIN like... 'SELECT * from spin1.patient JOIN cohortdefinitions.cohort_first25_spin1db ON patient.id=cohort_first25_spin1db.id' It fails. with errorCode=1, errorMessage='Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask' ANY IDEAS???? ... View more

Re: Does Atlas track Sqoop Exports?

by Explorer in Support Questions
‎03-14-2017 04:33 PM
‎03-14-2017 04:33 PM
Thanks for quick reply, got it. ... View more

Re: Does Atlas track Sqoop Exports?

by Explorer in Support Questions
‎03-14-2017 02:34 PM
‎03-14-2017 02:34 PM
Namit, looked at article you submitted. Are you saying that sqoop exports are not automatic like sqoop imports and therefore I should use the API or kafka topic to "work-around" this? ... View more

Does Atlas track Sqoop Exports?

by Explorer in Support Questions
‎03-09-2017 08:01 PM
2 Kudos
‎03-09-2017 08:01 PM
2 Kudos
Was able to follow the sandbox tutorial and got sqoop import to be tracked in atlas. I then made a new hive table by joining two tables I sqooped in and then tried to sqoop export the new hive table back to MySQL. This worked fine in terms of transferring the new hive table into a new MySQL table.... but Atlas did not track the export!?! (I can see the import lineage and the hiveQL lineage...but no export lineage.) So, does Altas support sqoop exports? or is their something else I have to configure. (using Atlas 0.7 and Hortonworks Sandbox 2.5) ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎05-08-2017 06:45 PM
Community Statistics
Member Since ‎03-09-2017 08:00 PM
Last Visited ‎05-08-2017 06:45 PM
Posts 6
Kudos received 5