Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (2)
Super Guru

MiniFi For Image Capture and Ingestion from Raspberry PI

16268-minifiimagesreceivingflow.png

Our Apache NiFi 1.2 instance is waiting for data to be pushed from MiniFi instance.

16270-minifgetimages.png

We export this to an XML template to convert to YAML.

minifi-toolkit-1.0.2.1.4.0-5/bin/config.sh transform MinifiImage.xml config.yml

Then I scp over the YAML file to my MiniFi RPI 3.

16269-minifiimagesclimessages.png

A clean running MiniFi instance. Not there will be some gzipped logs showing up in there.

16271-minifigetfileimages.png

Grab all the images that are available (make sure they are complete and at least 30 seconds old and bigger than 4096 bytes).

16272-minfirunschedule.png

Schedule when you want things to run. Our use case is one picture a minute.

16273-minifi-remoteprocessgroupsdetails.png

Make sure you map your remote process group to the correct Input Port!

MiniFi Setup

Make sure you have Oracle's JDK 8 installed and in the path.

export JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt/

bin/minifi.sh start

Check the Logs

tail -f log/minifi-app.log

Check the Status

bin/minifi.sh flowStatus controllerservices:health,bulletins

Test your Camera

raspistill -o cam4.jpg

Python Code

#!/usr/bin/python

# heavily remixed existing code
import os
import datetime
import traceback
import math
import random, string
import base64
import picamera
from time import sleep
from time import gmtime, strftime


def randomword(length):
 return ''.join(random.choice(string.lowercase) for i in range(length))
 
# Create unique image name
img_name = '/opt/demo/images/pi_image_{0}_{1}.jpg'.format(randomword(3),strftime("%Y%m%d%H%M%S",gmtime()))
 
# Capture Image from Pi Camera
try: 
 camera = picamera.PiCamera()
 camera.annotate_text = " Stored with Apache NiFi "
 camera.capture(img_name, resize=(800,600))


 pass
finally:
 camera.close()

Create an image using the Raspberry Pi official camera (I am using Noir). Add image text to each image. I place that script in a shell script and call it from MiniFi. I could use an execute script, I'll do that in the next one.

/opt/demo/runimage.sh

Using MiniFi to send over my images is easier (no weird conversions of binary data), safer, includes more provenance and is preferred over sending with MQTT. If my machine is too tiny (Onion Omega) or locked down, I would use MQTT.

References

1,758 Views
Comments
Expert Contributor

This is superb. I plan to do the same. Quick question , whats the version of the Minifi in the sense is it Java or C ++ . @Timothy Spann

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 12:26 PM
Updated by:
 
Contributors
Top Kudoed Authors