Has anyone installed the minifi C2 ( command and control ) . I need some steps to get this working. Do we have an installation document for this.

[#1]

Has anyone installed the minifi C2 ( command and control ) . I need some steps to get this working. Do we have an installation document for this. I have got my minifi agents on two separate servers which transmit data to the Nifi Server. I plan to create a C2 Server which will co-ordinate with the minifi agents installed on different machines. I have a high level idea of what C2 will do , but in the absence of any documents , its a bit difficult for me to progress. Need some help here .

@Vinicius Higa Murakami , @Shu , @Pierre Villard , @Matt Burgess , @Matt Clarke

Have you worked on this by any chance or provide any inputs to this.

[#2 · Accepted solution]

Hi,

i found the guidelines from "How to build an IIoT system using Apache NiFi, MiNiFi, C2 Server, MQTT and Raspberry Pi" written by Abdelkrim Hadjidj use full

the basic start command : "/bin/c2.sh", make sure you have the configuration correctly

i also found an Jira Issue solution for version 0.4

By removing the file javax.ws.rs-api-2.1.jar from lib directory in 0.4.0 version, it works
as expected.

[#3]

Thanks @Raymond Honderdors . I have started looking into it but i am still struggling to get this sorted. I am not sure which ingestor to use and where should the changed config YML file lie , whether on the C2 server or at the target server where minifi is installed.

[#4]

If you want to use the http ingestor

update the miNiFi ./conf/bootstrap.conf (C2 section "nifi.minifi.notifier")

update ./conf/minifi-c2-context.xml point the beam to the NiFi server that will hold the templates. Each ingestor has it own constructor.

create/import the template in NiFi

now start the c2 server and test "http://c2-server:10080/c2/config?class=iot-minifi-raspberry-agent&version=1" this should return the config.yml

now when you start miNiFi you will see in the logs that after a short interval miNiFi comes to live.

the how to in my first answer describes a step by step setup

[#5]

Thanks @Raymond Honderdors . Hey quick question . I think i need to also tweak the authorizations and authorizers.xml to add the CLASS. Do you have any details on that by any chance

[#6]

Hi @Abinav Joshi

If you mean in miNiFi, you need to set the bootstrap to point to the C2 Server

and set the class # Query string to pull configurations with (no version), "<yourflowname/templatename>" has to match exactly

nifi.minifi.notifier.ingestors.pull.http.query=class=<yourflowname/templatename>

authorizations and authorizers.xml would be a whole new topic, as it will deal with who can do what after sign-in

[#7]

Thanks @Raymond Honderdors. I think i am now close. I want to first try to test the c2 server. I have already run the script c2.sh and it looks up and running . My template on the remote nifi server is called FROM_CAPLTDA1E. So , if I want the config.yml returned , then would my url look like this ===> http://C2SERVER:8080/c2/config?class=FROM_CAPLTDA1E .

I guess this is the first step that needs to be sorted after which the minifi related settings can be tweaked. Can you please let me know if my approach is correct and also whether the URL mentioned above is right.

Many Many thanks @Raymond Honderdors by the way . The idea is much much clear now

[#8]

Hi @Raymond Honderdors

I started the C2 server and tried to test the URL below but it keeps on giving me the error --> HTTP ERROR 500

http://C2Server:8080/c2/config?class=CAPLTDA1E

where CAPLTDA1E is the name of the template on the nifi server. I have also updated minifi-c2-context.xml to ppint to the Nifi server. I am on version 0.4 for C2 and also removed the lib file javax.ws.rs-api-2.1.jar.

is there anything I am missing here ?

cheers

Abhi

[#9]

Hi, @Abhinav Joshi

are you sure you set the C2 Server port to be 8080, the default is 10080

besides that all looks to be fine, if it still does not work please share the snippets from your config

[#10]

Hi @Raymond Honderdors. Actually I tried using the 10080 port before but it seems this port is already used in my case which is why I was getting an error address already used.I then used 8080 and put that in the c2.properties file as well.

Please also note that my Nifi server is secured . The error that I am getting from my log file is as follows :-

Server returned HTTP response code: 403 for URL: https://capltda23:8443/nifi-api/flow/templates

DEBUG [qtp438135304-15] o.a.n.m.c2.provider.util.HttpConnector Connecting to endpoint: https://capltda23:8443/nifi-api/flow/templates

WARN [qtp438135304-15] o.a.nifi.minifi.c2.service.ConfigService Unable to get configuration. org.apache.nifi.minifi.c2.api.ConfigurationProviderException: Unable to retrieve template list

Didn't find any templates that matched \QFROM_CAPLTDA1E.v\E([0-9+])\Q\E

the above error states that C2 server couldn't get hold of the template name. Not sure why this is occuring.

Thanks again @Raymond Honderdors . I guess I am pretty close now . Let me know of your thoughts.

[#11]

Hey @Raymond Honderdors .. all good mate. This issue is now fixed. I had to rename my template to have the suffix .v1. I ignored that which is why I wasnt getting the config.yml . Its all sorted now and its working fine.

Thanks again for all your help

[#12]

Hi Abhinav,Did u receive below error while working on above implementation

o.a.n.m.b.c.i.PullHttpChangeIngestor Hit an exception while trying to pull java.net.ConnectException: Failed to connect to /127.0.0.1:10080

[#13]

@Vipin Chourasia I reckon this might be a firewall issue or something . I remember seeing something like this and then realized the firewall wasnt opened between the C2 server and the Minifi server

[#14]

I checked both the windows system ,firewall is open .checked by using ping coomand.

[#15]

Hi Abhinav, I am doing the same POC connect MINIFI NIFI using minifi c2 .Could you please guide us on this .As of now we have done minifi to nifi connectivity-Receiving data to nifi end , generated from minifi agent but now have to use minifi c2 controller.Please help buddy.

What i have understood Minifi c2 and nifi should be in same machine and minifi in another machine(windows)

then did some configuration changes and try to run minifi c2 but getting error .

I am following below link

https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi-ce1d...

[#16]

I follow the steps.

1. downloaded minif c2 server in windows where my nifi is running.

2.created a template using nifi ui with name given examplename iot-minifi-raspberry-agent.v1

3.changed minifi c2 property port -8080 (i was also getting port 10080 is already in use)

4.changed minifi-c2-context.xml property

----> <constructor-arg> <value>http://localhost:9090/nifi/</value> </constructor-arg> in this url my nifi is working,so this only changed

5.my minifi is running in different windows machine there i changed bootstrap property like below

---------------------------------------*****--------------

# Hostname on which to pull configurations from

nifi.minifi.notifier.ingestors.pull.http.hostname=localhost

# Port on which to pull configurations from

nifi.minifi.notifier.ingestors.pull.http.port=8080

# Path to pull configurations from

minifi.notifier.ingestors.pull.http.path=/c2/config

# Query string to pull configurations with

nifi.minifi.notifier.ingestors.pull.http.query=class=iot-minifi-raspberry-agent ----// name of my template ?? should i use version also like .v1

# Period on which to pull configurations from, defaults to 5 minutes if commented out nifi.minifi.notifier.ingestors.pull.http.period.ms=60000

-----------------------------------*****-------------------

6.now all changes are done now checking whether my changes are done correctly by hitting below url --

http://localhost:8080/c2/config?class=iot-minifi-raspberry-agent.v1&version=1

I can see yml is showing but that is already present in minifi c2 -->files folder ,

how my template's yml will create.where i m doing wrong or my understanding is incorrect .Please help @Raymond Honderdors @Abhinav Joshi @Bryan Bende

Any help would be really appreciated.Thanks in advance

[#17]

Hi @Vipin Chourasia ..apologies , for not being able to respond earlier . I have prepared a document to go through all this . Its a big document , can you give me your email id . I will flick it through .. I also went through the same issues but now our C2 is working very well.

[#18]

Thanks for reply Abhinav.You can't imagine how much relax i am feeling now .I was chashing you from many days.even i have tried to contact Abdelkrim Hadjidj for https://medium.freecodecamp.org/building-an-iiot-system-using-apache-nifi-mqtt-and-raspberry-pi-ce1d... // 

The current status of my POC is - I am able to create data flow between minifi(more than 2 system )and NIFI but stuck in C2 server configuration. I have done setup in windows machine.Thanks again Abhi

[#19]

I am struck at configuring the C2 server. Is it possible to share the document. 

[#20]

Hi @Vipin Chourasia . sorry mate . I know this is something that is very new and there is hardly any material out there . I also chased Abdelkrim 🙂 but didnt get any response. hahaha. Anyways .. sent you the document now. Have a read through