Use commodity hardware (Arduino Boards) to monitor temperature and humidity with NiFi.
get the source to follow along on git: REPO
In this tutorial we will need some hardware:
- Arduino Board
- DHT11 Sensor (DHT22 will work as well)
- Jumper wires
In Arduino IDE select your board and port and enter the following code:
#include "DHT.h"
#define DHTPIN 2 // what digital pin we're connected to
// Uncomment whatever type you're using!
#define DHTTYPE DHT11 // DHT 11
//#define DHTTYPE DHT22 // DHT 22 (AM2302), AM2321
//#define DHTTYPE DHT21 // DHT 21 (AM2301)
// Connect pin 1 (on the left) of the sensor to +5V
// NOTE: If using a board with 3.3V logic like an Arduino Due connect pin 1
// to 3.3V instead of 5V!
// Connect pin 2 of the sensor to whatever your DHTPIN is
// Connect pin 4 (on the right) of the sensor to GROUND
// Connect a 10K resistor from pin 2 (data) to pin 1 (power) of the sensor
// Initialize DHT sensor.
// Note that older versions of this library took an optional third parameter to
// tweak the timings for faster processors. This parameter is no longer needed
// as the current DHT reading algorithm adjusts itself to work on faster procs.
DHT dht(DHTPIN, DHTTYPE);
void setup() {
Serial.begin(9600);
dht.begin();
}
void loop() {
// Wait a few seconds between measurements.
delay(2000);
// Reading temperature or humidity takes about 250 milliseconds!
float h = dht.readHumidity();
// Read temperature as Fahrenheit (isFahrenheit = true)
float f = dht.readTemperature(true);
Serial.print(h);
Serial.print(", ");
Serial.print(f);
Serial.print("\n");
}
Flash/Upload the code to your arduino.
You should now be able to see the csv data being printed to the Serial Monitor in your Arduino IDE!
If you don't see any data check that you have the baud rate set to 9600.
If you still have issues verify the connections to your arduino are correct.
Now we need to set up a python application that will read the Serial data from the Arduino and convert it to a JSON format that we can use for stream processing in NiFi.
open up your favorite python IDE and create a file called arduino-nifi.py.
Enter the following code:
# -*- coding: utf-8 -*-
"""
Created on Thu Feb 22 15:54:50 2018
@author: vvagias
"""
import serial
import json
ser = serial.Serial('/dev/cu.usbmodem1411', 9600)
a = ser.readline().decode('utf8').replace("\n","").split(',')
js = {
"temperature" : float(a[1]),
"humidity" : float(a[0])
}
print(json.dumps(js))
We now have our data in valid JSON format and have also converted our string values for temperature and humidity to floats.
We are now ready to move on to NiFi.
Open NiFi and either create the flow in the image below or import the template included in the zip folder from git.
the Execute Process calls our python script and passes the data on to the Evaluate JSONPath processor.
It is set to pull the temperature out and place it on the flow file as an attribute. The RouteOnAttribute processor then routes the data based on the value of temperature being greater than 72 degrees.
This is where you can start using your imagination!!
Load a datadatabase, send alerts, load a dashboard (banana) perhaps.
Stay tuned for Data Science applications built on top of this foundation!
Hope you enjoyed this tutorial 🙂
