Minifi not connecting to Nifi - remote instance indicates that the port no longer exist

[#1]

HI Friends ,

I am trying to get minifi to trasmit data to Nifi however I keep on getting the following error :-

2018-09-12 10:38:41,913 ERROR [Timer-Driven Process Thread-3] o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=******,targets=https://******8443/nifi/] failed to communicate with ***:8443/nifi/ because the remote instance indicates that the port no longer exists

I have ensured that the SSL certificates are working , the firewalls are opened and Nifi is up and running but I am not sure why I keep getting this error "because the remote instance indicates that the port no longer exists" . I can see that my Nifi instance is up and running so I am not sure why minifi logs keep stating this ?

Also please note that the Nifi server is on version 1.5 and minifi on version 0.3

Any help here will be appreciated

thanks @Shu @Matt Clarke @Matt Burgess

[#2 · Accepted solution]

Hello @Abhinav Joshi!
Quick question, your INPUT PORT on the Nifi flow it's up?
Btw, I saw once this error from MiNifi when I changed my Input port in the Nifi flow and forgot to update the config.yml with the new ID from the Input Port.
Hope this helps

[#3]

Hi @Vinicius Higa Murakami Thanks for your response. I havent made any change as such . I will just show you the screen shots so that you can tell me whether I am on the same page

As seen above, are you referring to the Input Port like it is mentioned above . Basically i want to see data coming out from this Port to the log Attribute processor. The Id of the port is 7d8779b7-0164-1000-0000-00007fc774f5 as seen above,

However when I look at my YML file , under the section for input ports i.e in my case FROM_CADLBIG03 i dont see this number. Please see the section below , higlighted is the id number .

Do you think , I should update that id number with the input port id in the earlier screen shot ?

please let me know

[#4]

Hello @Abhinav Joshi!
Yes, change it in all references for this port.
E.g. in my config.yml I've 3 ref's to InputPort id

vmurakami$ cat config.yml | grep -i bcfad0b8-3cdb-15b8-0000-00004b94b9cf
  name: FetchFile/success/bcfad0b8-3cdb-15b8-0000-00004b94b9cf
  destination id: bcfad0b8-3cdb-15b8-0000-00004b94b9cf
  - id: bcfad0b8-3cdb-15b8-0000-00004b94b9cf

After that, restart minifi.
Hope this helps

[#5]

Thanks mate. I did that change and ensured it is consistent on those three reference points in the yml file. However when I ran the minifi script , I am getting some weird errors :-

Exception in thread "main" java.io.IOException: Unable to successfully transform the provided configuration at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1714) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.start(RunMiNiFi.java:1183) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values

One thing though I have noticed is that when I create the YML file , somehow the Input Port ID is not reflecting right . Some other input port id number comes up and this is also reflected at the same 3 places but it doesn't match which the one created on Nifi UI which is kind of strange, I tried searching that incorrect input port id on the nifi canvas , and I got a match actually , but that was linking to the template that I created for the YML file and not the actual input port .

I will have a look again and revert but thanks alot for your inputs. It has definitely given me a direction for sure.

cheers

Abhi

[#6]

Thanks mate. I did that change and ensured it is consistent on those three reference points in the yml file. However when I ran the minifi script , I am getting some weird errors :-

Exception in thread "main" java.io.IOException: Unable to successfully transform the provided configuration at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1714) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.start(RunMiNiFi.java:1183) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values

One thing though I have noticed is that when I create the YML file , somehow the Input Port ID is not reflecting right . Some other input port id number comes up and this is also reflected at the same 3 places but it doesn't match which the one created on Nifi UI which is kind of strange, I tried searching that incorrect input port id on the nifi canvas , and I got a match actually , but that was linking to the template that I created for the YML file and not the actual input port .

I will have a look again and revert but thanks alot for your inputs. It has definitely given me a direction for sure.

cheers

Abhi

[#7]

Hi @Abhinav Joshi!
Glad to know that I could be helpful.
So regarding your issue, this is strange..
Well, does it happen the same if you change the ID for the input ports manually (by changing it directly the config.yml)?
Hope this helps

[#8]

Thanks mate for the revert. Yes , I changed the input port in the config.yml manually so that it matches with the input port on the Nifi Canvas. However , when I do that , the minifi doesnt start and I get the issue org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values

Another interesting point I saw was that , when I installed Nifi and Minifi on my windows local laptop , the minifi YML file was correctly showing the input port Id unlike in the case above where it is all Linux. I am not sure if it is something to do with the versions of minifi and nifi . On linux , Nifi is on Version 1.5 and minifi on 0.3 where as the one i installed on windows has the Nifi version as 1.7 and minifi as 0.5 ..

I will continue to investigate none the less. thanks alot for being so helpful and taking your time out on this topic

[#9]

Hi @Abhinav Joshi!
Sorry, my late.
Hm, that's strange, in this case, I'd try to enable the debug level for bootstrap in the logback.xml.
Is it showing only this single message? Could you share the whole message, please?
Thanks

[#10]

Hey Mate - got it working 🙂 thanks for all your help . Actually i think I had not updated the config.yml in the right way. When I properly updated it , minifi worked and data was seen coming out from the input port .. Having said that , I am not sure why I need to manually change the input port id in the yml file to get this working. Ideally it should be done automatically. never the less, I appreciate your help in this case . At least its working