@hitaay HDF 2.0.1 ships with NiFi 1.0.0. HDF 2.1.1 ships with NiFi 1.1.0. You can see the list of all new features across HDF via the release notes here: https://docs.hortonworks.com/HDPDocuments/HDF2/HDF-2.1.0/bk_dataflow-release-notes/content/ch_hdf_relnotes.html. For NiFi specific information, the release notes for NiFi 1.1.0 can be found here: https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.1.0 The short list follows: Core Framework Improvements User Experience Improvements Updated Versions of Dependencies New or Improved Processors New Utility Security Improvements ... View more

You can leverage hdfs quotas, here's my answer from another thread https://community.hortonworks.com/questions/11446/hdfs-quota-is-there-a-gui-to-control-hdfs-name-and.html ... View more

@hitaay Yes. You can install NiFi. The only thing I recommend is to not use the Ambari server dedicated for HDP. While it can work to add Nifi as another service with a few tricks, you will collide with HDP in regard to support and upgrades. Avoid that. You can even have your own Ambari server for NiFi, but that is probably overkill and you won't get much benefit as you would get if you were truly installing HDF. Instead, you could use "How to install and start NiFi" installation instructions presented at https://nifi.apache.org/docs.html. In principle, you download an archived installation file, unarchive to your preferred location and start the NiFi service. I assume that you understand the consequences of collocating NiFi with other client services deployed on the edge nodes. It depends on how you plan to use those client services and NiFi, but they could compete for resources. ++++ If any of the responses in this thread was helpful, please vote/accept best response. ... View more

@hitaay Start here: http://docs.hortonworks.com/HDPDocuments/SS1/SmartSense-1.3.0/bk_installation/content/ambari_install.html It answers 1, 2 and 3. Yes, you can limit what and how often. Sensitive information can be also randomized. Regarding one of your concerns, be aware that Activity Analyzers deployed to the NameNodes in the cluster do not process any utilization data besides HDFS. Therefore, to process YARN, MapReduce, and Tez utilization data, another instance of the Activity Analyzer needs to be deployed to another node in the cluster, preferably on a non-master node. If any of the responses was helpful, please vote and accept as best answer. ... View more