Created on 09-02-2017 06:59 PM - edited 08-17-2019 11:23 AM
This article aims to show how to planning a Nifi Cluster following the best practices.
1) Hardware Provisioning
2) Hardware Considerations for HDF
- General Hardware
A key design point of NiFi is to use typical enterprise class application servers.
Hardware failure:
- Machine Class
A NiFi cluster consists of a single class of machine
Balanced NiFi Node:
In-rack backplane/Top-of-rack Switch:
NIFI is designed to take advantage of:
Most important hardware factors :
3) HDF Disk Partition Baseline
4) Disk Partitioning – Nifi Nodes (Repositories)
5) NiFi: Default Cluster Recommendation
When not provided with information to gauge the rate and complexity of data flow, start with a default cluster of three nodes. Three nodes are needed for HA by Zookeeper Quorum process.
The SKU is priced for cores, but it can be split up. So, a 16 core SKU can be split into 3 machines of 4 cores each. More cores per node will improve throughput (up to an extent).
So, starting cluster for, say, 50MB/s sustained throughput for average Flow is:
6) NiFi Clusters Scale Linearly
Created on 09-03-2017 11:17 AM
Have you checked Nifi throughput using Content Repo in a JBOD mode instead of Raid? Basically, let application decide for the distribution of data.
User | Count |
---|---|
763 | |
379 | |
316 | |
309 | |
270 |