Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi sizing

Highlighted

Nifi sizing

New Contributor

Hi,

I want to delpoy Nifi and I have some questions about the best practice deployment.

I have alot of data (around 100B records per day) so I need the best performance.

1. Should I need to install Nifi on VM or physic server?

2. Except the following Sizing Guide there is another one?

Thanks.

2 REPLIES 2

Re: Nifi sizing

Expert Contributor

@Ya ko

The sizing guide you linked is a great place to start.

Something similar which you can also take a look through is our documentation:

https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_planning-your-deployment/content/ch_hard...

For maximum performance NiFi should be installed on a physical server - NiFi is constrained by disk and network usage, so making sure you configure the 3 repositories and map them appropriately to independent disks is crucial in extracting highest throughput.

Re: Nifi sizing

New Contributor

I actually have worked with Cloudera NiFI SMEs that have helped me size out a standard configuration for some of their DOD customers.

Below are the specs of the cluster

 

3 Node Configuration Each Node with the Following2U Rackmount Chassis with Redundant Power Supply o16 x Total Xeon Scalable Processor Cores / 2.1GHzo64 GB High Performance 2666 MHz DDR3 ECC Registered Memory oRedundant OS Hard Drive Configurationo80 TB Enterprise Storage§ContentRepo:32TB Storage; 2 RAID 1 mount points (total 16 TB usable)§Flowfile Repo:16TB Storage; 1 RAID 1 mount point (total 8 TB usable)§Provenance: Repo: 16TB Storage; 1 RAID 1 mount point (total 8 TB usable)§Other HDF components: 16 TB Storage; 1 RAID 1 mount point (total 8 TB usable)o8 TB Enterprise Storage§Zookeeper:8 TB Storage; 1 RAID 1 mount point (total 4 TB usable)Dual Port 10/100/1000/10000 10GigE Network Adapter / SFP+ or RJ45 ConnectionIPMI / iKVMDedicated PortCentOS 7.x Installed for TestingPSSC Labs HPC Hardware Integration & Testing

 

 

Don't have an account?
Coming from Hortonworks? Activate your account here