problem with getFTP to putHDFS?

[#1]

i try to move files from FTP directory to HDFS but shows error in image issue.

i'm using windows nifi so in core-sit.xml and hdfs-site.xml are stored in to the local windows machine and directly give target directory as /tmp/target in HDFS

The files are queued from FTP and shows error in HDFS

Thanks in advance

[#2]

I have enclosed a copy of the flow, which shows the error. Can anyone help me out for this.

[#3]

Could you please provide logs in <NIFI dir>/logs/nifi-app.log?

[#4]

2016-05-31 12:23:02,353 ERROR [StandardProcessScheduler Thread-1] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1405) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode.access$100(StandardProcessorNode.java:89) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1243) ~[na:na] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_91] at java.lang.Thread.run(Unknown Source) [na:1.8.0_91] Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.get(Unknown Source) [na:1.8.0_91] at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1388) ~[na:na] ... 9 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_91] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1247) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1243) ~[na:na] ... 6 common frames omitted Caused by: java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:328) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:253) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:196) ~[na:na] at org.apache.nifi.processors.hadoop.PutHDFS.onScheduled(PutHDFS.java:177) ~[na:na] ... 15 common frames omitted

[#5 · Accepted solution]

Looking at:

java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at

I'd say that there is a network issue between your NiFi instance and your HDFS cluster. Did you correctly configure the processor with IP addresses and ports? Are you sure it is reachable from your NiFi instance?

[#6]

how to check this?

[#7]

As you said earlier i download files core-site.xml,hdfs-site.xml from ambari placed in the windows machine

and placed the path in "Hadoop Configuration Resources" and directory as "/tmp/dest/".

it shows error like this

[#8]

core-site and hdfs-site contain information regarding IP addresses to use to connect your cluster. Depending of what is in your configuration files, your NiFi instance may not be able to reach HDFS.

[#9]

i tried to change core-site.xml and hdfs-site.xml files sandbox.hortonworks.com:8020 to 192.168.1.158:8020

its working fine