Member since
11-10-2017
1
Post
0
Kudos Received
0
Solutions
11-10-2017
08:09 PM
@Matt Burgess Hi Matt, I am using nifi-1.4.0 with HDP 2.6. I was facing the same issue in PutHiveStreaming processor. I replaced it with the nar you provided. It is working fine when I'm writing to a non partitioned table. But when I tried to write it to a partitioned hive table, it throws the following error : PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841]
failed to process due to java.lang.RuntimeException:
java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be
writable. Current permissions are: rwx------; rolling back session: {} java.lang.RuntimeException: java.lang.RuntimeException: The
root scratch dir: /tmp/hive on HDFS should be writable. Current permissions
are: rwx------ I have provided all the necessary permissions to /tmp/hive folder, still I'm facing the same issue. Please let me know if I'm missing something. Thanks, Mohit Jain , @Matt Burgess I am using nifi 1.4.0 with HDP 2.6. I was facing the same error. I replaced the hive nar bundle with the one you provided. It is now working fine for the unpartitioned table. But when I'm trying to ingest on the partitioned table, it throws the following error: PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841]
failed to process due to java.lang.RuntimeException:
java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be
writable. Current permissions are: rwx------; rolling back session: {} java.lang.RuntimeException:
java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be
writable. Current permissions are: rwx------ at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:523) at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.createPartitionIfNotExists(HiveEndPoint.java:454) at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:318) at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEndPoint.java:278) at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoint.java:215) at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:192) at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.java:122) at
org.apache.nifi.util.hive.HiveWriter.lambda$newConnection$6(HiveWriter.java:237) at
java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at
java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: The root scratch dir:
/tmp/hive on HDFS should be writable. Current permissions are: rwx------ at
org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:613) at
org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:555) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:509) ... 11 common
frames omitted I have given all the permissions to the /tmp/hive folder, still it throws the same error. Please let me know if I'm missing something. Thanks, Mohit
... View more