I have 2 Raspberry Pi's. and I want to install NiFi on it.
On RPI 1: I have downloaded and installed nifi as it stated in the official Admin guide. When I start nifi, it says started but I was not able to access the nifi web ui. BUT after couple of days I needed to install nprobe and ntopng on pi, so I did install them (its a different story). Meanwhile, my nifi web ui started working. I dont no the reason why it worked, expect that I restarted the pi.
In the same way, now I want to have nifi on another pi.
On RPI 2: I have downloaded and installed nifi as it stated in the official Admin guide and restarted. When I start nifi, it says started but I was not able to access the nifi web ui. This is just like the issues I had faced last time. But I dont no how to fix it.
I dont want to do unnecessary things like installing nprobe or ntopng, since I dont need them now.
The only difference I find starting nifi on RPI 1 and 2 is:
On RPI 1: the directory called 'jetty' getting created after startup. ex: <nifi instal dir>/work/jetty/... BUT
On RPI 2: there is no directory called 'jetty' getting created after startup.
Expect this, there are no error in logs either.
Can some one please help to solve this issue.?
Hi Amar, we've been running NiFi on RPI2 for about half year by now. What you're seeing is an initial startup delay when things get unpacked and exploded in work directory. It can take a few minutes based on how slow your SD Card is. Tip: use a major branded one like SanDisk and choose a high-speed one (usually they tag it for 4K videos, 90MB/s write sustained). Otherwise sd card corruption is imminent, over time.
So the answer is simply wait a little :)
Running on RPI is worth an HCC article of its own, I might come up with something in the next week or two. For now, you can optimize things by making your NiFi install lean. Drop most of the modules that don't make sense on RPI. Here's what's in my lib dir:
nifi-api-0.6.0.jar nifi-documentation-0.6.0.jar nifi-framework-nar-0.6.0.nar nifi-html-nar-0.6.0.nar nifi-http-context-map-nar-0.6.0.nar nifi-jetty-bundle-0.6.0.nar nifi-kerberos-iaa-providers-nar-0.6.0.nar nifi-ldap-iaa-providers-nar-0.6.0.nar nifi-nar-utils-0.6.0.jar nifi-properties-0.6.0.jar nifi-provenance-repository-nar-0.6.0.nar nifi-runtime-0.6.0.jar nifi-scripting-nar-0.6.0.nar nifi-ssl-context-service-nar-0.6.0.nar nifi-standard-nar-0.6.0.nar nifi-standard-services-api-nar-0.6.0.nar nifi-update-attribute-nar-0.6.0.nar slf4j-api-1.7.12.jar
Also, we are testing on RPI3, which already shows major performance improvements
Hello @Andrew Grande
Thanks for your response. I have noted about the SD card. I have RPI3 with 15GB card. I will need to check the provider.
The problem I have is start up issue. But nothing is reported in logs either. This is the end of my logs:
As you pointed, I have already removed some of the .nar files which are not required and did start nifi. I waited for about 30mins after issuing the start command but web ui not functioning.
Also, we know anything about the work/jetty directory and why it is not getting created on my PI.? An article on PI is worth having.
A wild guess - did you by any chance remove the jetty nar bundle too? Please list the contents of your library dir.
I do have the jetty nar bundle. Here is my list.
There is a good new and a bad news as well.
Good news: I am able to access the web ui.
Bad news: It worked once and then for some reason I need to stop and start again, this time it does not start.
When I try to stop nifi, I get either "SocketTimeoutException" or the below thing happens: When either of this happens, I am sure that, it wont start next time when I start.
Then I don't no when, it may start or may not. I am very much uncertain, when It runs or not !!
Also, I have another question: I had implemented a simple flow: Generateflow ---> putFile. During I had got the below error.
I found this ticket: https://issues.apache.org/jira/browse/NIFI-754 , according to this, its not fixed in any of the nifi versions but if you hava a java version other than open jdk 7, it can be fixed. it is that I am correct?
and pls see this as well, https://issues.apache.org/jira/browse/NIFI-1076 , now I am confused about my previous statement and ticket ?
Any help with this uncertain functioning of web ui and this error can be much appreciated.?
First, there's absolutely no reason why you should be installing Open JDK7 on RPI. Use an official Oracle JDK8 for ARM. Here's what I have (I think that should take care of one of your issues):
java -version java version "1.8.0_65" Java(TM) SE Runtime Environment (build 1.8.0_65-b17) Java HotSpot(TM) Client VM (build 25.65-b01, mixed mode)
Next, be careful with GenerateFlow, it can overwhelm even a super-expensive enterprise server. Ensure you have some message limit and schedule configured on it, and, additionally, set up the backpressure controls on the connection coming out of it (limit # of messages or total size, maybe expire them). The reason is it will generate data as fast (actually faster) as the system can. Great stress test, or if you want to crash your server intentionally :)
Hello @Andrew Grande
I am not supposed to use jdk 8 due to other constraints related to work. But since I have 2 pi's. On one pi I am running jdk7 and 8 respectively. Below is my one of my pi's java:
when i had this problem https://issues.apache.org/jira/browse/NIFI-754 , I had installed open jdk 8. that solved the issue and web ui worked. Later when I stop and start, it's not working any more. as I said, its totally uncertain, or I might be doing something wrong. ? Either way there is nothing logs for me correct.
Amar, can you advise what OS and version you are running, and what limitation you have from running Oracle JDK on these devices?
I've got a half dozen Raspberry Pi 3b's running Raspbian GNU/Linux 8 (jessie) with the default Oracle JDK build 1.8.0_65-b17 and HDF184.108.40.206 with no optimisation quite happily, takes a couple of minutes to boot mind you.
Hello @Dan Chaffey,
My goal is install NIFI on RPI's and possibly test nifi clustering on Pi.
I have 2 RPI's and their respective os and software version are as follows:
1st Pi: RPI2 running Raspbian GNU/Linux 8(jessie) with arm open jdk 7 and Nifi 0.5.1
2nd Pi: RPI3 running Raspbian GNU/Linux 8(jessie) with arm open jdk 8: I had installed 7 first but later installed 8 due to the issue (https://issues.apache.org/jira/browse/NIFI-754), do you think, switching to jdk 8 is the right solution for this issue in the link?
Note: On my 2nd pi (RPI3) : I have 2 instances of Nifi running : nifi 0.5.1 and 0.6.1 (because I want to try clustering of nifi)
I don't have any limitations running jdk 7 or 8 as such but as per technology stack planed : I need to use jdk 7 since other softwares on pi might want to run on jdk 7.
Major issue I have on both the pi's is: I don't no when the nifi web ui will function or will not function, though nifi says it is running (../bin/nifi.sh [start / status]). Once in a while I will be able to access the running web ui and their is no guarantee that I can access the ui on my next start if I stop the server in between for any reason. More over there is nothing in the logs (nifi-app.logs and nifi-bootstrap.logs) to make some lead.
Also, (Sometimes) When I try to stop nifi, I get either "SocketTimeoutException" where the process is eventually killed or the below thing happens: When either of this happens, I am sure that, web ui won't function on my next start.
Are you running nifi on pi? if yes and you are able to run it without hassle, please do advice me accessing nifi's web ui. Much appreciated.
Thanks for all the information. Nifi web ui is working fine now, for me it takes around 30-40 mins for unpacking nars and make them available.
However, I have this issue https://issues.apache.org/jira/browse/NIFI-754, with openjdk 7 arm version. Please let me know, if you have any work around.?