question Re: How to load data from local system file to HDFS using Nifi in Archives of Support Questions (Read Only)

How to load data from local system file to HDFS using Nifi

narasimha_varma — Mon, 19 Aug 2019 10:37:36 GMT

I try to load data from my local system file into HDFS by using Getfile processor in Nifi, but it produces "invalid directory" error

Re: How to load data from local system file to HDFS using Nifi

MattWho — Mon, 09 Jan 2017 21:44:47 GMT

@Narasimma varman

Is your NiFi a single instance of NiFi or a NiFi cluster?

If it is a cluster, keep in my mind by default the GetFile processor will be running on every node in that cluster. The validate will also run on every node as well, so make sure the directory exists on all nodes.

Also make sure you have only specified a d directory path in the "Input Directory" property in GetFile. In you case, you should have only "/root/example" for that property.

The filename you wish to pickup should be specified in the "File Filter" property.

Matt

Re: How to load data from local system file to HDFS using Nifi

narasimha_varma — Wed, 11 Jan 2017 12:11:40 GMT

Yes, I am using single instance of Nifi

Re: How to load data from local system file to HDFS using Nifi

narasimha_varma — Mon, 19 Aug 2019 10:37:28 GMT

Properties Nifi GetFile processor

Re: How to load data from local system file to HDFS using Nifi

mburgess — Thu, 12 Jan 2017 02:24:45 GMT

I see that "Keep Source File" is set to false, if this processor has executed successfully once, then the file will be removed, and when it runs again, the file will no longer be there. Also since the File Filter is a regular expression, you may want to specify sample\.txt (note the backslash) to match the filename exactly (otherwise the period matches any character so a file called sample1txt would be found as well).

Re: How to load data from local system file to HDFS using Nifi

MattWho — Thu, 02 Feb 2017 03:32:09 GMT

@Narasimma varman

Make sure the user the NiFi process is running as on your server has the necessary permissions to access that directory path and remove files from it. Matt

Re: How to load data from local system file to HDFS using Nifi

narasimha_varma — Fri, 17 Feb 2017 13:44:13 GMT

Thanks, now I can load from local

Re: How to load data from local system file to HDFS using Nifi

narasimha_varma — Fri, 17 Feb 2017 13:44:24 GMT

Thanks, now I can load from local