Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

problem with getFTP to putHDFS?

Solved Go to solution
Highlighted

problem with getFTP to putHDFS?

Explorer

i try to move files from FTP directory to HDFS but shows error in image issue.

i'm using windows nifi so in core-sit.xml and hdfs-site.xml are stored in to the local windows machine and directly give target directory as /tmp/target in HDFS

The files are queued from FTP and shows error in HDFS

Thanks in advance

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: problem with getFTP to putHDFS?

Looking at:

java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at

I'd say that there is a network issue between your NiFi instance and your HDFS cluster. Did you correctly configure the processor with IP addresses and ports? Are you sure it is reachable from your NiFi instance?

View solution in original post

8 REPLIES 8
Highlighted

Re: problem with getFTP to putHDFS?

Explorer

4663-issue.png

I have enclosed a copy of the flow, which shows the error. Can anyone help me out for this.

Highlighted

Re: problem with getFTP to putHDFS?

Could you please provide logs in <NIFI dir>/logs/nifi-app.log?

Re: problem with getFTP to putHDFS?

Explorer

2016-05-31 12:23:02,353 ERROR [StandardProcessScheduler Thread-1] o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due to java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. java.lang.RuntimeException: Failed while executing one of processor's OnScheduled task. at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1405) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode.access$100(StandardProcessorNode.java:89) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1243) ~[na:na] at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_91] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_91] at java.lang.Thread.run(Unknown Source) [na:1.8.0_91] Caused by: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException at java.util.concurrent.FutureTask.report(Unknown Source) [na:1.8.0_91] at java.util.concurrent.FutureTask.get(Unknown Source) [na:1.8.0_91] at org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1388) ~[na:na] ... 9 common frames omitted Caused by: java.lang.reflect.InvocationTargetException: null at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91] at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_91] at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_91] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125) ~[na:na] at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1247) ~[na:na] at org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1243) ~[na:na] ... 6 common frames omitted Caused by: java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:530) ~[na:na] at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:494) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.checkHdfsUriForTimeout(AbstractHadoopProcessor.java:328) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.resetHDFSResources(AbstractHadoopProcessor.java:253) ~[na:na] at org.apache.nifi.processors.hadoop.AbstractHadoopProcessor.abstractOnScheduled(AbstractHadoopProcessor.java:196) ~[na:na] at org.apache.nifi.processors.hadoop.PutHDFS.onScheduled(PutHDFS.java:177) ~[na:na] ... 15 common frames omitted

Highlighted

Re: problem with getFTP to putHDFS?

Looking at:

java.nio.channels.UnresolvedAddressException: null at sun.nio.ch.Net.checkAddress(Unknown Source) ~[na:1.8.0_91] at sun.nio.ch.SocketChannelImpl.connect(Unknown Source) ~[na:1.8.0_91] at

I'd say that there is a network issue between your NiFi instance and your HDFS cluster. Did you correctly configure the processor with IP addresses and ports? Are you sure it is reachable from your NiFi instance?

View solution in original post

Highlighted

Re: problem with getFTP to putHDFS?

Explorer

how to check this?

Highlighted

Re: problem with getFTP to putHDFS?

Explorer

As you said earlier i download files core-site.xml,hdfs-site.xml from ambari placed in the windows machine

and placed the path in "Hadoop Configuration Resources" and directory as "/tmp/dest/".

it shows error like this

Highlighted

Re: problem with getFTP to putHDFS?

core-site and hdfs-site contain information regarding IP addresses to use to connect your cluster. Depending of what is in your configuration files, your NiFi instance may not be able to reach HDFS.

Highlighted

Re: problem with getFTP to putHDFS?

Explorer

i tried to change core-site.xml and hdfs-site.xml files sandbox.hortonworks.com:8020 to 192.168.1.158:8020

its working fine

Don't have an account?
Coming from Hortonworks? Activate your account here