Created 09-28-2018 12:12 AM
Has anyone installed the minifi C2 ( command and control ) . I need some steps to get this working. Do we have an installation document for this. I have got my minifi agents on two separate servers which transmit data to the Nifi Server. I plan to create a C2 Server which will co-ordinate with the minifi agents installed on different machines. I have a high level idea of what C2 will do , but in the absence of any documents , its a bit difficult for me to progress. Need some help here .
@Vinicius Higa Murakami , @Shu , @Pierre Villard , @Matt Burgess , @Matt Clarke
Have you worked on this by any chance or provide any inputs to this.
Created 10-03-2018 06:20 AM
Hi,
i found the guidelines from "How to build an IIoT system using Apache NiFi, MiNiFi, C2 Server, MQTT and Raspberry Pi" written by Abdelkrim Hadjidj use full
the basic start command : "/bin/c2.sh", make sure you have the configuration correctly
i also found an Jira Issue solution for version 0.4
By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0 version, it works as expected.
Created 10-03-2018 06:20 AM
Hi,
i found the guidelines from "How to build an IIoT system using Apache NiFi, MiNiFi, C2 Server, MQTT and Raspberry Pi" written by Abdelkrim Hadjidj use full
the basic start command : "/bin/c2.sh", make sure you have the configuration correctly
i also found an Jira Issue solution for version 0.4
By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0 version, it works as expected.
Created 10-04-2018 02:30 AM
Thanks @Raymond Honderdors . I have started looking into it but i am still struggling to get this sorted. I am not sure which ingestor to use and where should the changed config YML file lie , whether on the C2 server or at the target server where minifi is installed.
Created 10-04-2018 05:42 AM
If you want to use the http ingestor
update the miNiFi ./conf/bootstrap.conf (C2 section "nifi.minifi.notifier")
update ./conf/minifi-c2-context.xml point the beam to the NiFi server that will hold the templates. Each ingestor has it own constructor.
create/import the template in NiFi
now start the c2 server and test "http://c2-server:10080/c2/config?class=iot-minifi-raspberry-agent&version=1" this should return the config.yml
now when you start miNiFi you will see in the logs that after a short interval miNiFi comes to live.
the how to in my first answer describes a step by step setup
Created 10-08-2018 02:12 AM
Thanks @Raymond Honderdors . Hey quick question . I think i need to also tweak the authorizations and authorizers.xml to add the CLASS. Do you have any details on that by any chance
Created 10-08-2018 06:00 AM
Hi @Abinav Joshi
If you mean in miNiFi, you need to set the bootstrap to point to the C2 Server
and set the class # Query string to pull configurations with (no version), "<yourflowname/templatename>" has to match exactly
nifi.minifi.notifier.ingestors.pull.http.query=class=<yourflowname/templatename>
authorizations and authorizers.xml would be a whole new topic, as it will deal with who can do what after sign-in
Created 10-08-2018 09:29 PM
Thanks @Raymond Honderdors. I think i am now close. I want to first try to test the c2 server. I have already run the script c2.sh and it looks up and running . My template on the remote nifi server is called FROM_CAPLTDA1E. So , if I want the config.yml returned , then would my url look like this ===> http://C2SERVER:8080/c2/config?class=FROM_CAPLTDA1E .
I guess this is the first step that needs to be sorted after which the minifi related settings can be tweaked. Can you please let me know if my approach is correct and also whether the URL mentioned above is right.
Many Many thanks @Raymond Honderdors by the way . The idea is much much clear now
Created 10-09-2018 08:56 PM
I started the C2 server and tried to test the URL below but it keeps on giving me the error --> HTTP ERROR 500
http://C2Server:8080/c2/config?class=CAPLTDA1E
where CAPLTDA1E is the name of the template on the nifi server. I have also updated minifi-c2-context.xml to ppint to the Nifi server. I am on version 0.4 for C2 and also removed the lib file javax.ws.rs-api-2.1.jar.
is there anything I am missing here ?
cheers
Abhi
Created 10-10-2018 06:04 AM
Hi, @Abhinav Joshi
are you sure you set the C2 Server port to be 8080, the default is 10080
besides that all looks to be fine, if it still does not work please share the snippets from your config
Created 10-10-2018 07:45 PM
Hi @Raymond Honderdors. Actually I tried using the 10080 port before but it seems this port is already used in my case which is why I was getting an error address already used.I then used 8080 and put that in the c2.properties file as well.
Please also note that my Nifi server is secured . The error that I am getting from my log file is as follows :-
Server returned HTTP response code: 403 for URL: https://capltda23:8443/nifi-api/flow/templates
DEBUG [qtp438135304-15] o.a.n.m.c2.provider.util.HttpConnector Connecting to endpoint: https://capltda23:8443/nifi-api/flow/templates
WARN [qtp438135304-15] o.a.nifi.minifi.c2.service.ConfigService Unable to get configuration. org.apache.nifi.minifi.c2.api.ConfigurationProviderException: Unable to retrieve template list
Didn't find any templates that matched \QFROM_CAPLTDA1E.v\E([0-9+])\Q\E
the above error states that C2 server couldn't get hold of the template name. Not sure why this is occuring.
Thanks again @Raymond Honderdors . I guess I am pretty close now . Let me know of your thoughts.
Created 10-11-2018 08:37 PM
Hey @Raymond Honderdors .. all good mate. This issue is now fixed. I had to rename my template to have the suffix .v1. I ignored that which is why I wasnt getting the config.yml . Its all sorted now and its working fine.
Thanks again for all your help
Created 11-12-2018 06:08 PM
Hi Abhinav,Did u receive below error while working on above implementation
o.a.n.m.b.c.i.PullHttpChangeIngestor Hit an exception while trying to pull java.net.ConnectException: Failed to connect to /127.0.0.1:10080
Created 11-22-2018 09:53 AM
@Vipin Chourasia I reckon this might be a firewall issue or something . I remember seeing something like this and then realized the firewall wasnt opened between the C2 server and the Minifi server
Created 11-28-2018 09:07 AM
I checked both the windows system ,firewall is open .checked by using ping coomand.
Created 11-09-2018 12:44 PM
Hi Abhinav, I am doing the same POC connect MINIFI NIFI using minifi c2 .Could you please guide us on this .As of now we have done minifi to nifi connectivity-Receiving data to nifi end , generated from minifi agent but now have to use minifi c2 controller.Please help buddy.
What i have understood Minifi c2 and nifi should be in same machine and minifi in another machine(windows)
then did some configuration changes and try to run minifi c2 but getting error .
I am following below link
Created 11-09-2018 02:25 PM
I follow the steps.
1. downloaded minif c2 server in windows where my nifi is running.
2.created a template using nifi ui with name given examplename iot-minifi-raspberry-agent.v1
3.changed minifi c2 property port -8080 (i was also getting port 10080 is already in use)
4.changed minifi-c2-context.xml property
----> <constructor-arg> <value>http://localhost:9090/nifi/</value> </constructor-arg> in this url my nifi is working,so this only changed
5.my minifi is running in different windows machine there i changed bootstrap property like below
---------------------------------------*****--------------
# Hostname on which to pull configurations from
nifi.minifi.notifier.ingestors.pull.http.hostname=localhost
# Port on which to pull configurations from
nifi.minifi.notifier.ingestors.pull.http.port=8080
# Path to pull configurations from
minifi.notifier.ingestors.pull.http.path=/c2/config
# Query string to pull configurations with
nifi.minifi.notifier.ingestors.pull.http.query=class=iot-minifi-raspberry-agent ----// name of my template ?? should i use version also like .v1
# Period on which to pull configurations from, defaults to 5 minutes if commented out nifi.minifi.notifier.ingestors.pull.http.period.ms=60000
-----------------------------------*****-------------------
6.now all changes are done now checking whether my changes are done correctly by hitting below url --
http://localhost:8080/c2/config?class=iot-minifi-raspberry-agent.v1&version=1
I can see yml is showing but that is already present in minifi c2 -->files folder ,
how my template's yml will create.where i m doing wrong or my understanding is incorrect .Please help @Raymond Honderdors @Abhinav Joshi @Bryan Bende
Any help would be really appreciated.Thanks in advance
Created 11-22-2018 05:23 AM
Hi @Vipin Chourasia ..apologies , for not being able to respond earlier . I have prepared a document to go through all this . Its a big document , can you give me your email id . I will flick it through .. I also went through the same issues but now our C2 is working very well.
Created on
11-22-2018
07:02 AM
- last edited on
05-07-2021
01:09 PM
by
cjervis
Thanks for reply Abhinav.You can't imagine how much relax i am feeling now .I was chashing you from many days.even i have tried to contact Abdelkrim Hadjidj for https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi-ce1d... //
The current status of my POC is - I am able to create data flow between minifi(more than 2 system )and NIFI but stuck in C2 server configuration. I have done setup in windows machine.Thanks again Abhi
Created on
05-07-2021
11:23 AM
- last edited on
05-07-2021
01:10 PM
by
cjervis
I am struck at configuring the C2 server. Is it possible to share the document.
Created 11-22-2018 09:46 AM
Hi @Vipin Chourasia . sorry mate . I know this is something that is very new and there is hardly any material out there . I also chased Abdelkrim 🙂 but didnt get any response. hahaha. Anyways .. sent you the document now. Have a read through