How to load data from local system file to HDFS using Nifi

[#1]

I try to load data from my local system file into HDFS by using Getfile processor in Nifi, but it produces "invalid directory" error

[#2 · Accepted solution]

@Narasimma varman

Is your NiFi a single instance of NiFi or a NiFi cluster?

If it is a cluster, keep in my mind by default the GetFile processor will be running on every node in that cluster. The validate will also run on every node as well, so make sure the directory exists on all nodes.

Also make sure you have only specified a d directory path in the "Input Directory" property in GetFile. In you case, you should have only "/root/example" for that property.

The filename you wish to pickup should be specified in the "File Filter" property.

Matt

[#3]

Yes, I am using single instance of Nifi

[#4]

Properties Nifi GetFile processor

[#5]

I see that "Keep Source File" is set to false, if this processor has executed successfully once, then the file will be removed, and when it runs again, the file will no longer be there. Also since the File Filter is a regular expression, you may want to specify sample\.txt (note the backslash) to match the filename exactly (otherwise the period matches any character so a file called sample1txt would be found as well).

[#6]

Thanks, now I can load from local

[#7]

@Narasimma varman

Make sure the user the NiFi process is running as on your server has the necessary permissions to access that directory path and remove files from it. Matt

[#8]

Thanks, now I can load from local