It is fairly uncommon to virtualize environments for HDP / CDH.  However what you are proposing is possible.  We are actually working with the USAF and they are propsing a similar configuration to test different environments.    Also in my view you can save significant dollars looking at other hardware platforms than HPE Synergy. I'm definitely more than a bit biased since my companies develops and manufacturers server and cluster platforms for Cloudera platforms.  ... View more

Hi Yan:   Yes this documentation is terribly outdated.  However when we are building our clusters for our Cloudera customers now the hardware configuration is not so much different than previous versions.  There are more cores and larger disk drives but we are not doing anyhting exotic like Optane at this point.  I do think that with tiering it is probably time to consider an SSD and spinning disk platform.  Maybe consider NVMe drives but not really sure you are going to need that performance. I would bump up memory (at least 256GB per node at this point) before moving to NVMe. As for networking dual 10GigE is standard.  I think we will probably shoot through 25G / 40G and move right to 100G if pricing conintues to come down.    Alex   ... View more

I actually have worked with Cloudera NiFI SMEs that have helped me size out a standard configuration for some of their DOD customers. Below are the specs of the cluster   3 Node Configuration Each Node with the F ollowing • 2 U Rackmount Chassis with Redundant Power Supply o 16 x Total Xeon Scalable Processor Cores / 2. 1 GHz o 64 GB High Performance 2666 MHz DDR3 ECC Registered Memory o Redundant OS Hard Drive Configuration o 80 T B Enterprise Storage § Content Repo: 32 TB Storage ; 2 RAID 1 mount points (total 16 TB usable) § Flowfile Repo: 16 TB Storage ; 1 RAID 1 mount point (total 8 TB usable ) § Provenance: Repo : 16 TB Storage ; 1 RAID 1 mount point (total 8 TB usable) § Other HDF components: 16 TB Storage ; 1 RAID 1 mount point (total 8 TB usable) o 8 T B Enterprise Storage § Zookeeper: 8 TB Storage ; 1 RAID 1 mount point (total 4 TB usable) • Dual Port 10/100/1000/10000 10GigE Network Adapter / SFP+ or RJ45 Connection • IPMI / iKVM Dedicated Port • CentOS 7.x Installed for Testing • PSSC Labs HPC Hardware Integration & Testing     ... View more