question NiFi Hardware Recommendation in Archives of Support Questions (Read Only)

NiFi Hardware Recommendation

melvinmendoza — Fri, 09 Dec 2016 09:14:36 GMT

I have 700+ devices that generates 4MB ~ 1GB file per device (let's say per hour) , Is there any minimum hardware specs for production?

Re: NiFi Hardware Recommendation

bbende — Fri, 09 Dec 2016 20:32:38 GMT

This is an extremely rough guess and it really depends on a lot of factors, but if you assume all devices produce 1GB an hour thats 700GB/hour, thats about 200MBs/second. You are probably looking at a 7-8 node NiFi cluster, where each node has 16-24 cpus, 16+ GB RAM, 10 Gigabit NICs. You will also need to ensure proper separation of the repositories on different disks.

Re: NiFi Hardware Recommendation

melvinmendoza — Mon, 12 Dec 2016 11:36:03 GMT

@Bryan Bende estimate, average data collected is 10~20GB per day, lets say 20GB, it is safe to say to use a single node nifi?

Re: NiFi Hardware Recommendation

bbende — Mon, 12 Dec 2016 21:55:07 GMT

Yes that is a lot less than 700 devices generating 1GB each per hour 🙂

Most people want some kind of redundancy for production so you may still want to a smaller cluster, say 3 nodes, but yes you should be able to do that on one node.

Re: NiFi Hardware Recommendation

melvinmendoza — Thu, 15 Dec 2016 09:50:01 GMT

Great! thanks @Bryan Bende!