Processing Voice Data
Using a Raspberry Pi 3 with Google AIY Voice Kit is an easy way to control data flows in an organization.
The AIY Voice Kit is a good way to prototype voice ingest. It is not perfect, but for low end inexpensive hardware it is a good solution for those that speak clearly and are willing to repeat a phrase a few times for activation.
I was able to easily add a word for it to react to. I also have it waiting for someone to press the button to activate.
Setup and Run Steps
1. Install box
2. Setup GCP Account
3. As Pi User ~/bin/AIY-voice-kit-shell.sh src/assistant_library_demo.py
4. Download and Unzip Apache MiniFi
5. Custom Action controlx
6. Run it pi@raspberrypi:~/AIY-voice-kit-python/src $ ./controlx.py
7. start minifi
8. Apache MiniFi will tail the command file and look for audio files
Code Added to Google Example
| elif event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED and event.args: |
| print('You said:', event.args['text']) |
| text = event.args['text'].lower() |
| f=open("/opt/demo/commands.txt", "a+") |
| f.write(text) |
| f.close() |
| if 'process' in text : |
| self._assistant.stop_conversation() |
| self._say_ip() |
| elif 'computer' in text: |
| self._assistant.stop_conversation() |
| self._say_ip() |
| elif text == 'ip address' : |
| self._assistant.stop_conversation() |
| self._say_ip() |
| def _say_ip(self): |
| ip_address = subprocess.check_output("hostname -I | cut -d' ' -f1", shell=True) |
| f=open("/opt/demo/commands.txt", "a+") |
| f.write('My IP address is %s' % ip_address.decode('utf-8')) |
| f.close() |
| # aiy.audio.say('My IP address is %s' % ip_address.decode('utf-8')) |
| #aiy.audio.say('Turning on LED') |
| #voice_name = '/opt/demo/voices/voice_{0}.wav'.format(strftime("%Y%m%d%H%M%S",gmtime())) |
| #aiy.audio.record_to_wave(voice_name, 5) |
| aiy.voicehat.get_led().set_state(aiy.voicehat.LED.BLINK) |
| aiy.audio.say('MiniFy') |
| print('My IP address is %s' % ip_address.decode('utf-8')) |
Apache NiFi Ingest Flow
Flow File for a Sound File
Flow File For Commands
Apache MiniFi Ingest
Apache NiFi Overview
Text Command Processing
AIYVOICE Schema in Hortonworks Schema Registry
Source Code:
https://github.com/tspannhw/nifi-googleaiy
Results:
An example of text deciphered from my speaking into the AIY Voice Kit.
what was the process that we could try that a different way right now when was text put it in the file
My IP address is 192.168.1.199
The Recorded Audio Files Sent
../images/voice_20171229195641.wav
../images/voice_20171229202456.wav
../images/voice_20171229203028.wav
Inferred Schema
inferred.avro.schema
{ "type" : "record", "name" : "aiyvoice", "fields" : [ { "name" : "schema", "type" : "string", "doc" : "Type inferred from '\"aiyvoice\"'" }, { "name" : "path", "type" : "string", "doc" : "Type inferred from '\"./\"'" }, { "name" : "schemaname", "type" : "string", "doc" : "Type inferred from '\"aiyvoice\"'" }, { "name" : "commands0", "type" : "string", "doc" : "Type inferred from '\"process process process process process process process process processMy IP address is 192.168.1.199\n\"'" }, { "name" : "filename", "type" : "string", "doc" : "Type inferred from '\"commands.3088-3190.txt\"'" }, { "name" : "ssaddress", "type" : "string", "doc" : "Type inferred from '\"192.168.1.199:60896\"'" }, { "name" : "receivedFrom", "type" : "string", "doc" : "Type inferred from '\"AIY_GOOGLE_VOICE_PI\"'" }, { "name" : "sshost", "type" : "string", "doc" : "Type inferred from '\"192.168.1.199\"'" }, { "name" : "mimetype", "type" : "string", "doc" : "Type inferred from '\"text/plain\"'" }, { "name" : "tailfileoriginalpath", "type" : "string", "doc" : "Type inferred from '\"/opt/demo/commands.txt\"'" }, { "name" : "uuid", "type" : "string", "doc" : "Type inferred from '\"76232a9a-baf1-4f6b-9707-6d532653bbe8\"'" }, { "name" : "RouteOnAttributeRoute", "type" : "string", "doc" : "Type inferred from '\"commands\"'" } ] }Apache NiFi Created Apache Hive DDL
hive.ddl CREATE EXTERNAL TABLE IF NOT EXISTS aiyvoice (path STRING, filename STRING, commands0 STRING, ssaddress STRING, receivedFrom STRING, sshost STRING, mimetype STRING, tailfileoriginalpath STRING, uuid STRING, RouteOnAttributeRoute STRING) STORED AS ORC
Resources:
- https://github.com/google/aiyprojects-raspbian/blob/master/src/action.py
- https://github.com/google/aiyprojects-raspbian/blob/aiyprojects/HACKING.md
- https://aiyprojects.withgoogle.com/voice/#project-overview
- https://projects.raspberrypi.org/en/projects/google-voice-aiy
- https://github.com/googlesamples/assistant-sdk-python https://developers.google.com/assistant/sdk/guides/library/python/embed/run-sample
- http://ktinkerer.co.uk/play-bbc-radio-raspberry-pi-aiy-project/
- https://github.com/ktinkerer/aiyprojects-raspbian/tree/radio_player/src
- https://github.com/google/aiyprojects-raspbian
- https://github.com/n8kowald/raspi-voice-actions/blob/master/action.py
- https://github.com/n8kowald/raspi-voice-actions
- https://medium.com/@aallan/a-magic-mirror-powered-by-aiy-projects-and-the-raspberry-pi-e6a0fea3b4d6
