Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi user getting Tez exception

avatar
Super Collaborator

Hi,

We are experiencing a strange situation , my flow has 1 PutHDFS,5 PutHiveQL processes. When i use my user keytab it all works as expected but it fails when i use NiFi user keytab. That too on one particular PutHiveQL processor where it uses the following syntax.

insert overwrite table db_test.tbl_test PARTITION(date)

SELECT col1,col2,col3.....,col100 FROM temp_table where date= 20170717 DISTRIBUTE BY col100

getting this error

PutHiveQL[id=b1b830f3-d843-13d4-af26-f3e1aa74942d] Failed to update Hive for StandardFlowFileRecord[uuid=0992aeb1-4a00-4b40-b012-bdb055917ed2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1500476164378-589, container=default, section=589], offset=138, length=1463],offset=0,name=20170717.csv,size=1463] due to java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask; it is possible that retrying the operation will succeed, so routing to retry: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

Other 4 PutHiveQLs use ALTER TABLE Commands which works fine with NiFi user , only the above SQL is failing. But it works with my user id. All users are setup same in Ranger. only issue may be my user is created in all nodes where nifi user may not be..will that be causing this issue or anything else.??

Regards,

Sai

1 ACCEPTED SOLUTION

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
Super Collaborator
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

@Saikrishna Tarapareddy, can you please specify what folders and files needed the permissions? Are they on the HiveServer2 server or on HDFS?

avatar

I found it. In my case it was /user/nifi on HDFS, which is the user that the flow connects with.

It moves a .jar file under the .hiveJars subdirectory on that directory in order to run commands.