Created 05-06-2019 10:01 PM
Hello,
I see "permission denied" problem in PutHDFS with writing of file
"531fb09e-5599-48e2-b627-e558d1b6ba70-list"
org.apache.nifi.processor.exception.ProcessException: IOException thrown from PutHDFS[id=016a1008-69b2-1e66-6220-26211a37e1a5]: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/data/clustering/temp/.531fb09e-5599-48e2-b627-e558d1b6ba70-list":hdfs:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:353) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:325) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:246) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1950) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1934) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1917) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2763) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2698) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2582) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:736) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:409) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
It looks to me as "user=root" is the problem, how can I configure PutHDFS to execute as "user=hdfs"?
Thank you,
Created 05-07-2019 07:06 AM
The error indicates that you might be running "NiFI" server as "root" user and the "root" user does not have read/write permission on the HDFS directory "/data/clustering/temp/" where your NiFi processor is trying to write as the Dir permission on your HDFS is as "hdfs:hdfs:drwxr-xr-x":
So you may need to choose the options like Change the permission of "/data/clustering/temp/" HDFS directory recursively so that the "root" user cvan write on HDFS.
# su - hdfs -c "hdfs dfs -chown -R root:hadoop /data/"
.
Created 06-13-2019 03:29 AM
Did changing the permission help? Can you share your observations.