Support Questions

Find answers, ask questions, and share your expertise

problem with getFTP to putHDFS?

avatar
Rising Star

i try to move files from FTP directory to HDFS but shows error in image issue.

i'm using windows nifi so in core-sit.xml and hdfs-site.xml are stored in to the local windows machine and directly give target directory as /tmp/target in HDFS

The files are queued from FTP and shows error in HDFS

Thanks in advance

1 ACCEPTED SOLUTION

avatar

Looking at:

java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at

I'd say that there is a network issue between your NiFi instance and your HDFS cluster. Did you correctly configure the processor with IP addresses and ports? Are you sure it is reachable from your NiFi instance?

View solution in original post

8 REPLIES 8

avatar
Rising Star

4663-issue.png

I have enclosed a copy of the flow, which shows the error. Can anyone help me out for this.

avatar

Could you please provide logs in <NIFI dir>/logs/nifi-app.log?

avatar
Rising Star

2016-05-31 12:23:02,353 ERROR [StandardProcessScheduler Thread-1] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1405) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode.access$100(StandardProcessorNode.java:89) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1243) ~[na:na] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_91] at java.lang.Thread.run(Unknown Source) [na:1.8.0_91] Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.get(Unknown Source) [na:1.8.0_91] at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1388) ~[na:na] ... 9 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_91] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1247) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1243) ~[na:na] ... 6 common frames omitted Caused by: java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:328) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:253) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:196) ~[na:na] at org.apache.nifi.processors.hadoop.PutHDFS.onScheduled(PutHDFS.java:177) ~[na:na] ... 15 common frames omitted

avatar

Looking at:

java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at

I'd say that there is a network issue between your NiFi instance and your HDFS cluster. Did you correctly configure the processor with IP addresses and ports? Are you sure it is reachable from your NiFi instance?

avatar
Rising Star

how to check this?

avatar
Rising Star

As you said earlier i download files core-site.xml,hdfs-site.xml from ambari placed in the windows machine

and placed the path in "Hadoop Configuration Resources" and directory as "/tmp/dest/".

it shows error like this

avatar

core-site and hdfs-site contain information regarding IP addresses to use to connect your cluster. Depending of what is in your configuration files, your NiFi instance may not be able to reach HDFS.

avatar
Rising Star

i tried to change core-site.xml and hdfs-site.xml files sandbox.hortonworks.com:8020 to 192.168.1.158:8020

its working fine