Support Questions
Find answers, ask questions, and share your expertise

NiFi user getting Tez exception

Solved Go to solution
Highlighted

NiFi user getting Tez exception

Super Collaborator

Hi,

We are experiencing a strange situation , my flow has 1 PutHDFS,5 PutHiveQL processes. When i use my user keytab it all works as expected but it fails when i use NiFi user keytab. That too on one particular PutHiveQL processor where it uses the following syntax.

insert overwrite table db_test.tbl_test PARTITION(date)

SELECT col1,col2,col3.....,col100 FROM temp_table where date= 20170717 DISTRIBUTE BY col100

getting this error

PutHiveQL[id=b1b830f3-d843-13d4-af26-f3e1aa74942d] Failed to update Hive for StandardFlowFileRecord[uuid=0992aeb1-4a00-4b40-b012-bdb055917ed2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1500476164378-589, container=default, section=589], offset=138, length=1463],offset=0,name=20170717.csv,size=1463] due to java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask; it is possible that retrying the operation will succeed, so routing to retry: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

Other 4 PutHiveQLs use ALTER TABLE Commands which works fine with NiFi user , only the above SQL is failing. But it works with my user id. All users are setup same in Ranger. only issue may be my user is created in all nodes where nifi user may not be..will that be causing this issue or anything else.??

Regards,

Sai

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi user getting Tez exception

Super Collaborator

We fixed it , NiFi is missing permissions on the temp folders\files that it creates during INSERT..SELECT command.

View solution in original post

3 REPLIES 3
Highlighted

Re: NiFi user getting Tez exception

Super Collaborator

We fixed it , NiFi is missing permissions on the temp folders\files that it creates during INSERT..SELECT command.

View solution in original post

Highlighted

Re: NiFi user getting Tez exception

@Saikrishna Tarapareddy, can you please specify what folders and files needed the permissions? Are they on the HiveServer2 server or on HDFS?

Re: NiFi user getting Tez exception

I found it. In my case it was /user/nifi on HDFS, which is the user that the flow connects with.

It moves a .jar file under the .hiveJars subdirectory on that directory in order to run commands.

Don't have an account?