Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi GetHttp Fail to Process Session & Failed to Retrieve File from API,NiFi GetHttp Failed to Process Session Failed to Retrieve File from API

Highlighted

NiFi GetHttp Fail to Process Session & Failed to Retrieve File from API,NiFi GetHttp Failed to Process Session Failed to Retrieve File from API

New Contributor

I am getting issues with a basic processor in NiFi trying to get API JSON object

2016-06-29 02:22:28,296 INFO [StandardProcessScheduler Thread-7] o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled GetHTTP[id=522b5acf-57d5-4043-a680-8fbad42e5e9e] to run with 1 threads 2016-06-29 02:22:28,500 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec I/O exception (java.net.SocketException) caught when processing request to {}->http://finance.google.com:80: Invalid argument 2016-06-29 02:22:28,500 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec Retrying request to {}->http://finance.google.com:80 2016-06-29 02:22:28,575 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec I/O exception (java.net.SocketException) caught when processing request to {}->http://finance.google.com:80: Invalid argument 2016-06-29 02:22:28,575 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec Retrying request to {}->http://finance.google.com:80 2016-06-29 02:22:28,651 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec I/O exception (java.net.SocketException) caught when processing request to {}->http://finance.google.com:80: Invalid argument 2016-06-29 02:22:28,651 INFO [Timer-Driven Process Thread-2] org.apache.http.impl.execchain.RetryExec Retrying request to {}->http://finance.google.com:80 2016-06-29 02:22:28,711 INFO [Flow Service Tasks Thread-1] o.a.nifi.controller.StandardFlowService Saved flow controller org.apache.nifi.controller.FlowController@5338469 // Another save pending = false 2016-06-29 02:22:28,728 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.GetHTTP GetHTTP[id=522b5acf-57d5-4043-a680-8fbad42e5e9e] Failed to retrieve file from http://finance.google.com/finance/info?client=ig&q=YHOO due to Invalid argument; rolling back session: java.net.SocketException: Invalid argument 2016-06-29 02:22:28,729 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.GetHTTP java.net.SocketException: Invalid argument at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.7.0_45] at java.net.SocketInputStream.read(SocketInputStream.java:152) ~[na:1.7.0_45] at java.net.SocketInputStream.read(SocketInputStream.java:122) ~[na:1.7.0_45] at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) ~[httpcore-4.4.1.jar:4.4.1] at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) ~[httpclient-4.4.1.jar:4.4.1] at org.apache.nifi.processors.standard.GetHTTP.onTrigger(GetHTTP.java:433) ~[nifi-standard-processors-0.6.1.jar:0.6.1] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1059) [nifi-framework-core-0.6.1.jar:0.6.1] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-0.6.1.jar:0.6.1] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-0.6.1.jar:0.6.1] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:123) [nifi-framework-core-0.6.1.jar:0.6.1] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_45] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_45] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_45] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_45] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_45] at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] 2016-06-29 02:22:28,730 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.GetHTTP GetHTTP[id=522b5acf-57d5-4043-a680-8fbad42e5e9e] Failed to process session due to org.apache.nifi.processor.exception.ProcessException: java.net.SocketException: Invalid argument: org.apache.nifi.processor.exception.ProcessException: java.net.SocketException: Invalid argument

Any ideas on how to resolve this?

I am running NiFi locally and on a remote server. Similar issues with failed session and file retrieval happen.

Thank you,

Julian

3 REPLIES 3
Highlighted

Re: NiFi GetHttp Fail to Process Session & Failed to Retrieve File from API,NiFi GetHttp Failed to Process Session Failed to Retrieve File from API

Could you post a screenshot of your processor config?

Highlighted

Re: NiFi GetHttp Fail to Process Session & Failed to Retrieve File from API,NiFi GetHttp Failed to Process Session Failed to Retrieve File from API

Master Guru

@Julian Hernandez Rather then using the getHTTP processor, try using the invokeHTTP to perform a GET against the http://finance.google.com/finance/info?client=ig&q=YHOO URL. The "response" relationship of the invokeHTTP processor should contain the content you are looking for.

Matt

Highlighted

Re: NiFi GetHttp Fail to Process Session & Failed to Retrieve File from API,NiFi GetHttp Failed to Process Session Failed to Retrieve File from API

New Contributor

Matt,

I appreciate your prompt feedback. The Hortonworks team looked into the problem and somehow the GetHttp resolved itself, not sure how. They also showed me how to use the invokeHTTP, which started throwing a few "yield" errors that resolved themselves (very weird).

Thank you for your suggestion,

Julian

Don't have an account?
Coming from Hortonworks? Activate your account here