use NiFi processor to parse files mounted on file system NFS

[#1]

Hello,

I need to parse logs mounted on file system NFS on centos.

Is NiFi reliable with NFS ? with files processors

Thanks,

Regards

[#2 · Accepted solution]

@Bertrand Goubot

A mounted file systems is treated like any other local system directory NiFi interacts with. NFS mounts are not going to be as performant as local disks of course. When NiFi ingest Files form your NFS mount, the content of those files is going to be placed in NiFi's content repository. Any processors that then work on that ingested content work use what is in teh content repository.

Thanks,

Matt

[#3]

@Bertrand Goubot

MiNiFi will work the same way with regards to accessing locally mounted disks.

NiFi and MiNiFi have no issues working with large Files as long as there is sufficient space in the content repository to store and do any processing needed on those large files.

Thanks,

Matt TIP: We try to keep the discussion going under one answer rather then creating a new answer every time we respond back and forth in this forum. Unless someone if offering up a new solution/answer to the question.

[#4]

thanks Matt for your valuable response (and tip too :))

[#5]

Hello MattWho,

Some time spent since your reply, you would still recommend to mount the NFS drives on each NiFi nodes - or are there any further development on this topic?

[#6]

Thanks Matt for your quick response.

Do you know if MiNiFi works as same with NFS mount ?

content is going to be placed in NiFi's content repository : but how NiFi (or MiNiFi) deals with large files, bigger thant 1Gb ?

Thanks, Regards.