question Re: Problem working with Hive Tables using python code in Support Questions

Problem working with Hive Tables using python code

New_To_Hive — Mon, 02 Oct 2023 13:38:57 GMT

I was trying to use python query to extract data from Hive table and create a data frame. It gave me the following error at cursor.execute line.

pyhive.exc.OperationalError: TExecuteStatementResp(status=TStatus(statusCode=3, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask:17:16', 'org.apache.hive.service.cli.operation.Operation:toSQLException:Operation.java:348', 'org.apache.hive.service.cli.operation.SQLOperation:runQuery:SQLOperation.java:228', 'org.apache.hive.service.cli.operation.SQLOperation:runInternal:SQLOperation.java:265', 'org.apache.hive.service.cli.operation.Operation:run:Operation.java:260', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatementInternal:HiveSessionImpl.java:575', 'org.apache.hive.service.cli.session.HiveSessionImpl:executeStatement:HiveSessionImpl.java:550', 'org.apache.hive.service.cli.CLIService:executeStatement:CLIService.java:285', 'org.apache.hive.service.cli.thrift.ThriftCLIService:ExecuteStatement:ThriftCLIService.java:567', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1557', 'org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement:getResult:TCLIService.java:1542', 'org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39', 'org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39', 'org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor:process:HadoopThriftAuthBridge.java:647', 'org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286', 'java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1149', 'java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:624', 'java.lang.Thread:run:Thread.java:748'], sqlState='08S01', errorCode=1, errorMessage='Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask'), operationHandle=None)

Any resolution will be great.

Re: Problem working with Hive Tables using python code

cjervis — Mon, 02 Oct 2023 13:45:27 GMT

Welcome to the community @New_To_Hive. While you wait form someone more knowledgable to reply, I thought I could at least point something out. I see the following

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask:17:16

I suggest double checking your Hive query for correctness and reviewing your Tez logs to see if they hold additional clues.

Re: Problem working with Hive Tables using python code

ggangadharan — Tue, 21 Nov 2023 17:37:34 GMT

The error you're encountering (OperationalError: TExecuteStatementResp(status=TStatus(statusCode=3, ...) indicates that there was an issue during the execution of the Hive query. The specific error message within the response is Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask.

Here are a few steps you can take to troubleshoot and resolve the issue:

Check Hive Query Logs:
- Review the Hive query logs to get more details about the error. The logs might provide information about the specific query or task that failed, including any error messages or stack traces.
- You can find the logs in the Hive logs directory. The location may vary based on your Hadoop distribution and configuration.
Inspect Query Syntax:
- Double-check the syntax of your Hive SQL query. Ensure that the query is valid and properly formed. Sometimes, a syntax error can lead to execution failures.
Verify Hive Table Existence:
- Confirm that the Hive table you're querying actually exists. If the table or the specified database is missing, it can lead to errors.
Check Permissions:
- Verify that the user running the Python query has the necessary permissions to access and query the Hive table. Lack of permissions can result in execution errors.
Examine Tez Configuration:
- If your Hive queries use the Tez execution engine, check the Tez configuration. Ensure that Tez is properly configured on your cluster and that there are no issues with the Tez execution.
Look for Resource Constraints:
- The error message mentions TezTask, so consider checking if there are any resource constraints on the Tez execution, such as memory or container size limitations.
Update Python Library:
- Ensure that you are using a compatible version of the Python library for interacting with Hive (e.g., pyhive or pyhive[hive]). Updating the library to the latest version might help resolve certain issues.
Test with a Simple Query:
- Simplify your query to a basic one and see if it executes successfully. This can help isolate whether the issue is specific to the query or a more general problem.

After reviewing the logs and checking the mentioned aspects, you should have more insights into what might be causing the error. If the issue persists, consider providing more details about the Hive query and the surrounding context, so we can offer more targeted assistance.