We are experiencing a strange situation , my flow has 1 PutHDFS,5 PutHiveQL processes. When i use my user keytab it all works as expected but it fails when i use NiFi user keytab. That too on one particular PutHiveQL processor where it uses the following syntax.
insert overwrite table db_test.tbl_test PARTITION(date)
SELECT col1,col2,col3.....,col100 FROM temp_table where date= 20170717 DISTRIBUTE BY col100
getting this error
PutHiveQL[id=b1b830f3-d843-13d4-af26-f3e1aa74942d] Failed to update Hive for StandardFlowFileRecord[uuid=0992aeb1-4a00-4b40-b012-bdb055917ed2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1500476164378-589, container=default, section=589], offset=138, length=1463],offset=0,name=20170717.csv,size=1463] due to java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask; it is possible that retrying the operation will succeed, so routing to retry: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
Other 4 PutHiveQLs use ALTER TABLE Commands which works fine with NiFi user , only the above SQL is failing. But it works with my user id. All users are setup same in Ranger. only issue may be my user is created in all nodes where nifi user may not be..will that be causing this issue or anything else.??
I found it. In my case it was /user/nifi on HDFS, which is the user that the flow connects with.
It moves a .jar file under the .hiveJars subdirectory on that directory in order to run commands.