Hadoop and Nifi integration best practices

[#1]

Hello Everyone

I would like to learn the best practices to integrate Nifi with Hadoop cluster. Please share the same.

Regards

[#2 · Accepted solution]

@spdvnz

NiFi's Hadoop based processors already include the Hadoop client libraries so there is no need to install them outside of NiFi or install NiFi on the same hardware where Hadoop is running. The various NiFi processors for communicating with Hadoop use the core-site.xml, hdfs-site.xml, and/or hbase-site.xml files as part of their configuration. These files would need to be copied from you Hadoop system(s) to a local directory on each of your NiFi instances for use by these processors.

Detailed processor documentation is provided by clicking on "help" in the upper right corner within the NiFi UI. You can also get to the processor documentation by right clicking on a processor and slecting "usage" from teh context menu that is displayed.

Thanks,

Matt