NiFi: Hadoop Configuration Error

[#1]

We have integrated NiFi service (HDF service) to our existing HDP cluster. NiFi installation was successful and NiFi is running without any error. But there is an error with PutHDFS. It's showing that there is an error in our Hadoop configuration. I'm posting the error and screenshot too.

2018-06-20 12:00:14,246 WARN [StandardProcessScheduler Thread-6] org.apache.hadoop.conf.Configuration /tmp/core-site.xml:an attempt to override final parameter: fs.defaultFS;  Ignoring.
2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] o.apache.nifi.processors.hadoop.PutHDFS PutHDFS[id=11a40827-0164-1000-ffff-ffffb07a04d9] HDFS Configuration error - java.net.ConnectException: Connection refused: {}

java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
        at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:345)
        at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:260)
        at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:205)
        at sun.reflect.GeneratedMethodAccessor728.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
        at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
        at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1334)
        at org.apache.nifi.controller.StandardProcessorNode$1.call(StandardProcessorNode.java:1330)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2018-06-20 12:00:14,248 ERROR [StandardProcessScheduler Thread-6] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally
java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException: null

[#2]

Hey @Abhinav Phutela !
Are you able to telnet the 50070 or other Datanode ports of those hosts (hdfs hosts)?
I made a test here, you can try to use as an example:

Content of GetFile Processor

Content of the PutHDFS

My flow above will send the hdfs_nifi.txt on the Nifi host and send to HDFS host.

#First on the HDFS side, need to send the *-site.xml config from the hadoop-client/conf dir to Nifi host
[root@hdfs_node1 conf]$ scp core-site.xml hdfs-site.xml root@nifi_node1:/home/nifi
#Then I'll spread the *-site.xml config to my other nifi instances and create the file to put on HDFS
[nifi@nifi_node1 ~]$ echo "testing hdfs content" > hdfs_nifi.txt
[nifi@nifi_node1 ~]$ sudo chown nifi:hadoop *.xml
[nifi@nifi_node1 ~]$ scp core-site.xml hdfs-site.xml nifi@nifi_node2:/home/nifi
[nifi@nifi_node1 ~]$ scp core-site.xml hdfs-site.xml nifi@nifi_node3:/home/nifi

#After i start the job, got the succesfully data on HDFS
[root@hdfs_node1 conf]$ hdfs dfs -cat /tmp/hdfs_nifi.txt
testing hdfs content

Ps: I also had changed my /etc/hosts to known the hosts from HDFS cluster.

Hope this helps!

[#3]

What should be the permission of the hdfs-site.xml and core-site.xml files which I'm supposed to copy in /home/nifi folder?

And I'm able to telnet to datanode and namenode both (50075 and 50070)