Thanks @Raymond Honderdors. I think i am now close. I want to first try to test the c2 server. I have already run the script c2.sh and it looks up and running . My template on the remote nifi server is called FROM_CAPLTDA1E. So , if I want the config.yml returned , then would my url look like this ===> http://C2SERVER:8080/c2/config?class=FROM_CAPLTDA1E . I guess this is the first step that needs to be sorted after which the minifi related settings can be tweaked. Can you please let me know if my approach is correct and also whether the URL mentioned above is right. Many Many thanks @Raymond Honderdors by the way . The idea is much much clear now ... View more

Thanks @Raymond Honderdors . Hey quick question . I think i need to also tweak the authorizations and authorizers.xml to add the CLASS. Do you have any details on that by any chance ... View more

Thanks @Raymond Honderdors . I have started looking into it but i am still struggling to get this sorted. I am not sure which ingestor to use and where should the changed config YML file lie , whether on the C2 server or at the target server where minifi is installed. ... View more

Hey @Vinicius Higa Murakami .. I now get it . The issue was with the minifi 0.3 tool kit . When i tried to create the config.yml file from minifi tool kit version 0.3 , i kept on getting this issue. As soon as I switched to the latest version for tool kit , this problem got fixed and the port id was correct. Thanks for the help. I have accepted your answer as well ... View more

Actually I recreated the entire process all over again and each time I have had to manually change the input port id. I have also discovered one thing. The XML file also looks fine and has the input port id defined. The change in the input port id seems to happen when the conversion takes place to YML file. I am using the minifi toolkit version 0.3 . Have you seen this kind of behavior before by any chance ? Anyways - I will accept your answer as the concept is clear now and this may just be a bug . Thanks alot for the help 🙂 ... View more

Hey Mate - got it working 🙂 thanks for all your help . Actually i think I had not updated the config.yml in the right way. When I properly updated it , minifi worked and data was seen coming out from the input port .. Having said that , I am not sure why I need to manually change the input port id in the yml file to get this working. Ideally it should be done automatically. never the less, I appreciate your help in this case . At least its working ... View more

Hi @Srinatha Anantharaman - we actually got in touch with our Hadoop admin and opened all firewalls for the thrift metastore server after which it worked, Hope this helps ... View more

Thanks mate for the revert. Yes , I changed the input port in the config.yml manually so that it matches with the input port on the Nifi Canvas. However , when I do that , the minifi doesnt start and I get the issue org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values Another interesting point I saw was that , when I installed Nifi and Minifi on my windows local laptop , the minifi YML file was correctly showing the input port Id unlike in the case above where it is all Linux. I am not sure if it is something to do with the versions of minifi and nifi . On linux , Nifi is on Version 1.5 and minifi on 0.3 where as the one i installed on windows has the Nifi version as 1.7 and minifi as 0.5 .. I will continue to investigate none the less. thanks alot for being so helpful and taking your time out on this topic ... View more

Thanks mate. I did that change and ensured it is consistent on those three reference points in the yml file. However when I ran the minifi script , I am getting some weird errors :- Exception in thread "main" java.io.IOException: Unable to successfully transform the provided configuration at org.apache.nifi.minifi.bootstrap.RunMiNiFi.performTransformation(RunMiNiFi.java:1714) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.start(RunMiNiFi.java:1183) at org.apache.nifi.minifi.bootstrap.RunMiNiFi.main(RunMiNiFi.java:244) Caused by: java.io.IOException: mapping values One thing though I have noticed is that when I create the YML file , somehow the Input Port ID is not reflecting right . Some other input port id number comes up and this is also reflected at the same 3 places but it doesn't match which the one created on Nifi UI which is kind of strange, I tried searching that incorrect input port id on the nifi canvas , and I got a match actually , but that was linking to the template that I created for the YML file and not the actual input port . I will have a look again and revert but thanks alot for your inputs. It has definitely given me a direction for sure. cheers Abhi ... View more