getfile : nifi does not have sufficient permissions

[#1]

Hello,

Why Nifi GetFile processor need write acces to the source directory ?

Directory is 755 chmoded and nifi do have read & execute acces, but not write acces.

Is it a bug ?

Here is the Error MSG :

2017-02-17 22:41:14,587 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.GetFile GetFile[id=4d57350a-015a-1000-ffff-fffffb56570e] GetFile[id=4d57350a-015a-1000-ffff-fffffb56570e] failed to process session due to java.lang.IllegalStateException: Directory '/tmp/test' does not have sufficient permissions (i.e., not writable and readable): java.lang.IllegalStateException: Directory '/tmp/test' does not have sufficient permissions (i.e., not writable and readable) 2017-02-17 22:41:14,588 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.GetFile

The version is : HDF 2.1.0

NiFi - Version 1.1.0.2.1.0.0-165

Best.

Adel.

[#2 · Accepted solution]

GetFile will delete the processed file by default. Check the Keep Source File setting, for example. https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.GetFile/index....

A better alternative for processing files without touching them is a combination of ListFile/FetchFile processors. These will maintain the state internally in NiFi to track the processed files.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ListFile/index...

[#3]

ListFile/FetchFile works fine, will do with that.

GetFile needs write permissions even if we need only to read from sources.

Thx.

[#4]

Please review the Keep Source File setting. The default is false, which means by default NiFi will attempt to delete the file from the directory after creating the associated FlowFile from it.

[#5]

Hello @slachterman and @Andrew Grande,

Thank's, but i already put "Keep Source File" to True, so i still do not understand why NIFI/Getfile needs write acces (does Nifi/GetFile processor needs to create ".state-file" ... in the source directory ?)

FYI ; i also disabled the "Recurse Subdirectories".

I'll try the FetchFile but GetFile matches more with my need.

Any other ideas ?

Thx.

[#6]

ListFile/FetchFile works fine, will do with that.

GetFile needs write permissions even if we need only to read from sources.

Thx.

[#7]

I've tried adding ListFile/FetchFile in a number of combinations and still get the same error. @adel gacem please could you elaborate on what the solution was that worked for you?

In this tutorial/example there don't seem to be any permission errors with GetFile, even with "Keep Source File" set to False.

@Andrew Grande the link you posted above (2017-02-17) no longer works: it seems now to have been replaced by single page containing all functions (incl GetFile) at https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/

[#8]

The solution for me was to chmod 777.

@adel gacem 755 is not enough.