Support Questions
Find answers, ask questions, and share your expertise

problem using tailFile for over 5G file?

Explorer

Hi

I have a python code to make a file with squid content (for example 50,000,000 and 5.9G)

I wanna transfer it HDFS durung runnig my code, so I use tailFile, mergeContent (Maximum Number of Entries = 10000) and putHDFS(block.size=64 MB)

cpu=8

ram=23G

heap size(in bootstrap.conf) = 4096MB

thread=6

After my code was finished, tailFile start to get all the content of my file but I am wanting to start when my code start .

I am using single NiFi

How can I solve it?

96403-tail.png

0 REPLIES 0