Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

NiFi Hardware Recommendation

avatar
Contributor

I have 700+ devices that generates 4MB ~ 1GB file per device (let's say per hour) , Is there any minimum hardware specs for production?

1 ACCEPTED SOLUTION

avatar
Master Guru

This is an extremely rough guess and it really depends on a lot of factors, but if you assume all devices produce 1GB an hour thats 700GB/hour, thats about 200MBs/second. You are probably looking at a 7-8 node NiFi cluster, where each node has 16-24 cpus, 16+ GB RAM, 10 Gigabit NICs. You will also need to ensure proper separation of the repositories on different disks.

View solution in original post

4 REPLIES 4

avatar
Master Guru

This is an extremely rough guess and it really depends on a lot of factors, but if you assume all devices produce 1GB an hour thats 700GB/hour, thats about 200MBs/second. You are probably looking at a 7-8 node NiFi cluster, where each node has 16-24 cpus, 16+ GB RAM, 10 Gigabit NICs. You will also need to ensure proper separation of the repositories on different disks.

avatar
Contributor

@Bryan Bende estimate, average data collected is 10~20GB per day, lets say 20GB, it is safe to say to use a single node nifi?

avatar
Master Guru

Yes that is a lot less than 700 devices generating 1GB each per hour 🙂

Most people want some kind of redundancy for production so you may still want to a smaller cluster, say 3 nodes, but yes you should be able to do that on one node.

avatar
Contributor

Great! thanks @Bryan Bende!