I am pretty new to Nifi and am experimenting. I have a head scratching problem I just can't figure out. This is my setup.
I have one virtual Ubuntu 18.04 server (Call it server A) that is running Nifi 1.9.2. The server has openjdk-8 installed. Nifi runs okay. I am experimenting with a setup where I request a file from a web server that is in JSON format. This works with no problem.
Meanwhile I have another virtual Ubuntu 18.04 server ( Server B ) that is running ThingsBoard. My plan is to use Thingsboard to see things within the NiFi data flow.
Getting these two to exchange data is another challenge.
Approach 1 - I configured Nifi to PublishMQTT to ThingsBoard. No success.
Approach 2 - I confgured Nifi to InvokeHTTP to ThingsBoard. No success.
Yet I have successfully demonstrated that -
1. Nifi on A can send MQTT to server B and be received by Mosquitto_sub.
2. Server A can run up Mosquitto and publish a message to server B and ThingsBoard receives it.
I haven't figured out how I can prove HTTP is getting over or not.
I recognise that Mosquitto will conflict with Nifi and Thingsboard if everyone is on port 1883, so I set up Mosquitto to default to 1884, then use the -p 1883 switch when I am experimenting.
I have gone through configurations, read countless web pages, tried anything I can think of. Using netstat I see that the network accessing is from JAVA. Therefore I tried exchanging JAVA implementations and versions. This didn't improve the situation. Oracle 11 and NiFi wouldn't start up. Only openjdk-8 seemed to satisfy it.
UFW isn't running, the virtual machines are on the same network, there are no routers or firewalls to configure.
Ok, fixed it. The problem was the original JSON had a construct error. NiFi didn't care and passed it through. ThingsBoard did care and rejected the whole packet. Once I had realised the problem I fixed the JSON using EvaluateJSONPath and ReplaceText and it all burst into life. Result.