Ingest your brainwaves with Apache NIFI

[#1]

Mindwave Neurosky

The Mindwave Neurosky is a headset that allows you to record your brainwaves using EEG technology. In this article we show you how to ingest these brainwaves with NIFI

Mindwave Neurosky driver installation for OSX Sierra

1. Download and install the latest driver from http://download.neurosky.com/public/Products/MindWave%20headset/RF%20driver%20for%20Mac/MindWaveDriv...
2. After the driver is installed, download and install the latest MindWave Manager from http://download.neurosky.com/public/Products/MindWave%20headset/RF%20driver%20for%20Mac/MindWave%20M...
3. Launch the MindWave Manager, navigate to "Pairing" section and click the "Search for MindWave", then follow the instructions to pair the headset.

Install NIFI on OSX Sierra with Homebrew

1. Install Homebrew from the terminal: /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
2. Install NIFI (time of writing v1.1.2): brew install nifi

Import NIFI Flow Template

An example flow template can be downloaded using Curl:

curl -O https://gist.githubusercontent.com/wardbekker/a80cbe7d12bc1866f393c5a74bf417a0/raw/9d64daa748dec352e...

The most important processor here is the ListenTCP processor, which will listen on port 20000 and will receive the JSON payload. The flow also contains a Site 2 Site NIFI connection to a remote processgroup with the URL http://wbekkerhdf0.field.hortonworks.com:9090/nifi. You can change it to your own remote NIFI cluster.

Get Ruby 'forward' script

The Mindwave Thinkgear driver will create a socket where we can consume the sensor data as Json messages. To ingest it with the current vanilla version of NIFI, we need to 'forward' the messages from the thinkgear port to the NIFI ListenTCP processor port number. Upcoming versions of NIFI will have a GetTCP processor, making this Ruby script obsolete.

Save this Ruby script as a file under thinkgear.rb. Run it with ruby thinkgear.rb AFTER you have connected your headset AND started ListenTCP processor on the NIFI flow. Otherwise you will run into connection errors.

require 'socket'
require 'json'
require 'date'
thinkgear_server_socket = TCPSocket.new 'localhost', 13854
nifi_server_socket = TCPSocket.new 'localhost', 20000
# trigger json output
thinkgear_server_socket.puts "{\"enableRawOutput\": true, \"format\": \"Json\"}\n"
while line = thinkgear_server_socket.gets # Read lines from socket
  hash = JSON.parse(line)
  hash['timestamp'] = DateTime.now.strftime('%Q')
  hash['user_id'] = 1
  json = JSON.generate(hash)
  puts json
  nifi_server_socket.puts json
end
thinkgear_server_socket.close
nifi_server_socket.close

Start ingestion of your brainwaves

1. Connect you headset by launching the MindWave Manager, navigate to "Pairing" section and click the "Search for MindWave", then follow the instructions to pair the headset.
2. Start the NIFI flow, or at least the ListenTCP processor.
3. Start the ruby script with ruby thinkgear.rb.

At this point you should see JSON output from your Mindwave headset on your terminal, and new flowfiles into NIFI. Have fun with your brainwaves!