Created 10-26-2022 01:19 PM
I have a Nifi process where I create a Hive table and add a single row to the table in a separate PutHiveQL process. The 'create table' process works most of the time, but the 'insert' almost always fails with the below error. This even though I direct the issue to the RetryFlowFile processor and repeat it 5 times. Note that I can run the same insert statement in Hue without any issues. What could be the problem? Is there anything I can tweak to make this work?
PutHiveQL[id=27ab3890-0b98-1ba9-8499-5c4985902ebf] Failed to update Hive for StandardFlowFileRecord[uuid=91a0e7ea-5f2d-4530-bcd8-6e650597f6c7,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1666794893345-1, container=default, section=1], offset=5660, length=128],offset=0,name=91a0e7ea-5f2d-4530-bcd8-6e650597f6c7,size=128] due to Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask; it is possible that retrying the operation will succeed, so routing to retry: java.sql.SQLException: Error while compiling statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
Created 10-28-2022 06:15 AM
@Vickey You will need to investigate the differences in execution from nifi vs hue. You should be able to see some differences in tez between both queries. The return code 1 is often due to tez resource availability. Also, i would suggest that you NOT do anything with retry logic in nifi until you are sure the processor works as expected the first time. Bringing in the retry logic only hides the issue if it is intermittent. Sounds like it is consistently not working, so retry is not helpful. Another suggestion is to turn the process logging level to DEBUG. Last but not least, in future, try to screen shot your flow, and processor configs so all of us can see more details
Created 10-31-2022 12:30 PM
@Vickey Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks
Regards,
Diana Torres,