Community Articles

TimothySpann · ‎01-28-2018

So I found another low-end affordable platform from China for running MiniFi goodness. For this 512MB RAM machine, I decided to use MiniFi CPP.

git clone https://github.com/apache/nifi-minifi-cpp.git
apt-get install cmake   gcc g++   bison   flex   libcurl-dev   librocksdb-dev librocksdb4.1   uuid-dev uuid   libboost-all-dev libssl-dev   libbz2-dev liblzma-dev   doxygen -y
apt-get install -y libleveldb-dev
apt-get install -y libxml2
apt-get install libpython3-dev -y
apt-get install liblua5.1-0-dev -y
apt-get install libusb-1.0.0-0-dev libpng12-dev -y
apt-get install docker.io python-virtualenv -y
 apt-get install libpython3-dev -y
apt-get install libgps-dev -y
apt-get install libpcap-dev -y
apt-get install cmake   gcc g++   bison   flex -y 
./bootstrap.sh
interactive UI
cd nifi-minifi-cpp-0.3.0-source/
mkdir build
cd build
cmake ..
make
make package
apt-get install libssl-dev 
-- The following features have been enabled:
 * EXPRESSION LANGUAGE EXTENSIONS , This enables NiFi expression language
 * HTTP CURL , This enables RESTProtocol, InvokeHTTP, and the HTTPClient for Site to Site
 * ROCKSDB REPOS , This Enables persistent provenance, flowfile, and content repositories using RocksDB
 * ARCHIVE EXTENSIONS , This Enables libarchive functionality including MergeContent, CompressContent, (Un)FocusArchiveEntry and ManipulateArchive.
 * SCRIPTING EXTENSIONS , This enables scripting
-- The following OPTIONAL packages have been found:
 * LibRt
 * Git
 * BZip2
 * LibLZMA
 * EXPAT
 * Boost
 * Doxygen
-- The following REQUIRED packages have been found:
 * BISON
 * FLEX
 * CURL
 * Threads
 * PythonLibs
 * ZLIB
 * UUID
 * OpenSSL
-- The following features have been disabled:
 * CIVETWEB , This enables ListenHTTP
-- The following OPTIONAL packages have not been found:
 * WinSock
 * RocksDB
 * LibArchive
 * Nettle
 * LibXml2

This will take some library installs for development and various libraries needed for networking, security and devices.

From the interactive UI, I selected to add most of the goodies except a USB camera since I don't have one on this tiny machine.

The CPP is different (and much smaller) than the Java one. For first, you have a specific set of processors, some of which are pretty cool like ones for USB Camera image ingestion, TensorFlow processing and other device goodness. You can browse the list in the PROCESSORs link below.

Since I built mine from the git clone of the master, I am running the 0.4.0 branch.

I could not install RocksDB on OrangePi

There's some cool stuff for reporting status.

root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0# bin/minifi
minifi            minificontroller  minifi.sh
root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0# bin/minifi.sh start
Starting MiNiFi with PID 15831 and pid file /opt/demo/nifi-minifi-cpp-0.4.0/bin/.minifi.pid
root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0# [2018-01-infol 16:24:17.591] [main] [info] Using MINIFI_HOME=/opt/demo/nifi-minifi-cpp-0.4.0 from environment.
[2018-01-infol 16:24:17.592] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties
[2018-01-26 16:24:17.893] [main] [info] Loading FlowController
[2018-01-26 16:24:17.893] [org::apache::nifi::minifi::FlowController] [info] Load Flow Controller from file /opt/demo/nifi-minifi-cpp-0.4.0/conf/config.yml
[2018-01-26 16:24:17.895] [org::apache::nifi::minifi::FlowController] [info] Loaded root processor Group
[2018-01-26 16:24:17.895] [org::apache::nifi::minifi::FlowController] [info] Initializing timers
[2018-01-26 16:24:17.896] [org::apache::nifi::minifi::FlowController] [info] Loaded controller service provider
[2018-01-26 16:24:17.896] [org::apache::nifi::minifi::FlowController] [info] Loaded flow repository
[2018-01-26 16:24:17.896] [org::apache::nifi::minifi::FlowController] [info] Starting Flow Controller
[2018-01-26 16:24:17.898] [org::apache::nifi::minifi::core::controller::StandardControllerServiceProvider] [info] Enabling % controller services
[2018-01-26 16:24:17.899] [org::apache::nifi::minifi::c2::C2Agent] [info] Class is RESTSender
[2018-01-26 16:24:17.902] [org::apache::nifi::minifi::io::Socket] [error] Could not bind to socket
[2018-01-26 16:24:17.903] [org::apache::nifi::minifi::FlowController] [info] Started Flow Controller
[2018-01-26 16:24:17.903] [main] [info] MiNiFi started
root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0/bin#  ./minificontroller --list components
[2018-01-infol 16:25:16.461] [controller] [info] MINIFI_HOME is not set; determining based on environment.
[2018-01-infol 16:25:16.462] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi.properties
[2018-01-infol 16:25:16.463] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties
Components:
FlowController
root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0/bin#    ./minificontroller --list connections
[2018-01-infol 16:25:32.850] [controller] [info] MINIFI_HOME is not set; determining based on environment.
[2018-01-infol 16:25:32.851] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi.properties
[2018-01-infol 16:25:32.852] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties
Connection Names:
 ./minificontroller --updateflow "config yml"
root@orangepizero:/opt/demo/nifi-minifi-cpp-0.4.0/bin#    ./minificontroller --getfull
[2018-01-infol 16:26:13.296] [controller] [info] MINIFI_HOME is not set; determining based on environment.
[2018-01-infol 16:26:13.297] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi.properties
[2018-01-infol 16:26:13.298] [org::apache::nifi::minifi::Properties] [info] Using configuration file located at /opt/demo/nifi-minifi-cpp-0.4.0/conf/minifi-log.properties
0 are full

References

https://github.com/apache/nifi-minifi-cpp

https://cwiki.apache.org/confluence/display/MINIFI/C2+Design+Proposal

https://github.com/apache/nifi-minifi-cpp/blob/master/examples/BidirectionalSiteToSite/README.md

https://nifi.apache.org/minifi/getting-started.html

https://github.com/apache/nifi-minifi-cpp/blob/master/PROCESSORS.md#

https://github.com/apache/nifi-minifi-cpp/blob/master/EXPRESSIONS.md

https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Versioncpp-0.3.0

https://github.com/apache/nifi-minifi-cpp/blob/master/Extensions.md

To Customize C++ Builds

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74685143

Build Extensions with MiniFi C++