question Re: Has anyone tried using a Apache NiFi on Raspberry Pi? in Archives of Support Questions (Read Only)

Has anyone tried using a Apache NiFi on Raspberry Pi?

ssen — Fri, 25 Sep 2015 21:40:46 GMT

Raspberry Pi is a very inexpensive system often hooked up to variety of sensors. It will be useful to use Raspberry Pi as the host for Apache NiFi to ingest and coordinate the data from sensors before transporting the stream to alerting system or persistent storage

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

andrewg — Fri, 25 Sep 2015 23:52:56 GMT

According to Joe W this was working. There are a few variants of RPI, so don't expect wonders, but it's absolutely possible for a scaled down instance with correct expectations.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

wsalazar — Sat, 26 Sep 2015 00:23:33 GMT

I have a Beagle Board Black that I can try it on. Any use case ideas?

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

pwendorf — Tue, 29 Sep 2015 19:24:36 GMT

Rasbian (default Pi OS) ships with the 1.8 JDK, so it should work fine.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

landon_t_robins — Thu, 10 Dec 2015 02:19:04 GMT

I've tried HDP on a Pi. Works pretty good. Will add Nifi to my list of things to POC...

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

nsabharwal — Thu, 10 Dec 2015 02:22:29 GMT

@Landon Robinson Do you have a doc or your own cheat sheet? If so then it would nice to have an article published here.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

landon_t_robins — Thu, 10 Dec 2015 02:29:12 GMT

I have some notes but nothing formal cobbled together. Most stuff goes on hadoopsters.net when I learn about it. I'm hoping to publish some more on Falcon here very soon. The article feature makes it easy.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

JoeWitt — Thu, 10 Dec 2015 02:33:05 GMT

Have run on both BeagleBone Black and the latest Rapsberry PI (not zero). Works quite well but Pi is considerably faster. It is slow to start as we load a lot of classes in the beginning but on a Pi type system you can certainly remove extraneous Nars. I think you can expect around 30MB/s of throughput and about 10K events/s with full features activated.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

rgelhausen — Fri, 11 Dec 2015 02:39:54 GMT

NiFi works on the Raspberry Pi without any custom configuration. Follow the normal "Getting Started" steps here. One thing to know is that the large number of standard NARs means "service nifi start" will take a long time (5-10 minutes) to complete. Once it is done, the web UI will function normally.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

mgrabenstein — Tue, 02 Feb 2016 04:49:43 GMT

Share would be great. I have one built, just have to image the micro SD cards.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

amar_cs06 — Thu, 17 Mar 2016 18:50:34 GMT

I doing the similar task but web UI is not functioning.

I have downloaded NIFI on to my Raspberry Pi 2 and run the ./nifi.sh start command (which is not returning prompt back, I waited for 10-15 mins, then after 1 enter click I get my terminal back to run other commands.). when I run "status" command it says NIFI is running currently.

However if I try to stop NIFI, I get "Waiting Apache Nifi to finish shutting down..." then "NIFI has not finished shutting down after 20 seconds. Killing process" and it stops. But web UI is not functioning. NO errors in logs.

Can some one please help with this.

Thanks,

Amar

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

amar_cs06 — Thu, 17 Mar 2016 20:45:48 GMT

@Randy Gelhausen @Saptak Sen, @Andrew Grande, @pwendorf, @jwitt

[just to notify you]

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

rgelhausen — Fri, 18 Mar 2016 05:08:41 GMT

Hi @Amar ch, I didn't time it, but it takes somewhere from 15-30 minutes to fully start the NiFi process.

One of the first things NiFi does on startup is unpack all the NAR files in the lib directory to make them available as processors. If you want the service to start faster, you can remove NARs for processors you don't intend to use.

Even once the service is started, I've found the NCM will be very slow. For this reason, I would plan on developing a template on your laptop and importing it into your RaspberryPi NiFi instance rather than trying to build out the flow directly on the pi.

The stumbling block above aside, once the flow is defined and started, I have had 0 problems with it. Been monitoring WiFi traffic for several months without a blip, even after a few power outages. NiFi started back up and resumed working just fine.

Edit: The number of standard NARs has grown since my comment in December, hence the increase in startup time from 10-30 minutes 😃

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

amar_cs06 — Wed, 13 Apr 2016 19:51:51 GMT

Hello @Randy Gelhausen

Thanks for your information. Indeed it took 30-40 mins for Nifi Web ui to be functional, after excluding all the nar that aren't required for now.

I wan't to know if you are using nifi in the clustered environment?

Thanks,

Amar.

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

TimothySpann — Wed, 13 Apr 2016 20:51:07 GMT

I am going to try on my PineA64 which is a little beefier

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

markdoutre — Tue, 26 Apr 2016 01:53:06 GMT

Was toying with having a go at this but came to the conclusion it wasn't worth the effort. I have lots of small processor based sensors like Arduino that are happily running MQTT. They all talk to a broker on a Pi.

Instead I just wrote a simple Python based MQTT to Kafka bridge and have the Pi publishing to a topic directly in HDP. Flume then sends the data to HDFS

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

TimothySpann — Thu, 16 Jun 2016 21:47:32 GMT

It worked on my raspberry pi 2 but it is a bit slow

Re: Has anyone tried using a Apache NiFi on Raspberry Pi?

r_young — Mon, 21 Nov 2016 19:21:30 GMT

Hi @Landon Robinson @jwitt i was trying to install ambari-agent on the pi3 but was having issues as the repo didnt support armv71. is there a way around this, thanks