Support Questions
Find answers, ask questions, and share your expertise

Processing large text files through Nifi returns null

New Contributor

I'm having a pretty weird issue.

I currently have a nifi flow which uses a getfile processor to grab a log file that is dropped in a directory. From there, its passed to a custom processor where the file is processed inside of session.read using the BufferedReader class. I do all of my necessary processing and pass the flow files on. Simple stuff

This works perfectly fine for moderately sized files but when I try to process a large log file (around 2.5GB) I start getting null returned when trying to call readLine() from buffered reader. It seems as if very large files aren't being opened/read by the bufferedreader.

Are large text files treated any differently in nifi compared to smaller files?