Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Guru

Processing Voice Data


Using a Raspberry Pi 3 with Google AIY Voice Kit is an easy way to control data flows in an organization.

The AIY Voice Kit is a good way to prototype voice ingest. It is not perfect, but for low end inexpensive hardware it is a good solution for those that speak clearly and are willing to repeat a phrase a few times for activation.

I was able to easily add a word for it to react to. I also have it waiting for someone to press the button to activate.

45636-45569-googlevoicekit.jpg


Setup and Run Steps

1. Install box

2. Setup GCP Account

3. As Pi User ~/bin/AIY-voice-kit-shell.sh src/assistant_library_demo.py

4. Download and Unzip Apache MiniFi

5. Custom Action controlx

6. Run it pi@raspberrypi:~/AIY-voice-kit-python/src $ ./controlx.py

7. start minifi

8. Apache MiniFi will tail the command file and look for audio files


Code Added to Google Example


elif event.type == EventType.ON_RECOGNIZING_SPEECH_FINISHED and event.args:
print('You said:', event.args['text'])
text = event.args['text'].lower()
f=open("/opt/demo/commands.txt", "a+")
f.write(text)
f.close()
if 'process' in text :
self._assistant.stop_conversation()
self._say_ip()
elif 'computer' in text:
self._assistant.stop_conversation()
self._say_ip()
elif text == 'ip address' :
self._assistant.stop_conversation()
self._say_ip()

def _say_ip(self):
ip_address = subprocess.check_output("hostname -I | cut -d' ' -f1", shell=True)
f=open("/opt/demo/commands.txt", "a+")
f.write('My IP address is %s' % ip_address.decode('utf-8'))
f.close()
# aiy.audio.say('My IP address is %s' % ip_address.decode('utf-8'))
#aiy.audio.say('Turning on LED')
#voice_name = '/opt/demo/voices/voice_{0}.wav'.format(strftime("%Y%m%d%H%M%S",gmtime()))
#aiy.audio.record_to_wave(voice_name, 5)
aiy.voicehat.get_led().set_state(aiy.voicehat.LED.BLINK)
aiy.audio.say('MiniFy')
print('My IP address is %s' % ip_address.decode('utf-8'))


Apache NiFi Ingest Flow

45638-aiyingestflow.png

Flow File for a Sound File

45639-aiywaveflowfile.png

Flow File For Commands

45640-aiyflow.png

Apache MiniFi Ingest

45641-aiyminififlow.png

Apache NiFi Overview

45642-aiyoverviewflow.png

Text Command Processing

45643-aiyrouteonattribute.png

AIYVOICE Schema in Hortonworks Schema Registry

45644-aiyschema.png


Source Code:

https://github.com/tspannhw/nifi-googleaiy

Results:

An example of text deciphered from my speaking into the AIY Voice Kit.

what was the process that we could try that a different way right now when was text put it in the file

My IP address is 192.168.1.199

The Recorded Audio Files Sent

../images/voice_20171229195641.wav

../images/voice_20171229202456.wav

../images/voice_20171229203028.wav


Inferred Schema

inferred.avro.schema
{ "type" : "record", "name" : "aiyvoice", "fields" : [ { "name" : "schema", "type" : "string", "doc" : "Type inferred from '\"aiyvoice\"'" }, { "name" : "path", "type" : "string", "doc" : "Type inferred from '\"./\"'" }, { "name" : "schemaname", "type" : "string", "doc" : "Type inferred from '\"aiyvoice\"'" }, { "name" : "commands0", "type" : "string", "doc" : "Type inferred from '\"process process process process process process process process processMy IP address is 192.168.1.199\n\"'" }, { "name" : "filename", "type" : "string", "doc" : "Type inferred from '\"commands.3088-3190.txt\"'" }, { "name" : "ssaddress", "type" : "string", "doc" : "Type inferred from '\"192.168.1.199:60896\"'" }, { "name" : "receivedFrom", "type" : "string", "doc" : "Type inferred from '\"AIY_GOOGLE_VOICE_PI\"'" }, { "name" : "sshost", "type" : "string", "doc" : "Type inferred from '\"192.168.1.199\"'" }, { "name" : "mimetype", "type" : "string", "doc" : "Type inferred from '\"text/plain\"'" }, { "name" : "tailfileoriginalpath", "type" : "string", "doc" : "Type inferred from '\"/opt/demo/commands.txt\"'" }, { "name" : "uuid", "type" : "string", "doc" : "Type inferred from '\"76232a9a-baf1-4f6b-9707-6d532653bbe8\"'" }, { "name" : "RouteOnAttributeRoute", "type" : "string", "doc" : "Type inferred from '\"commands\"'" } ] }


Apache NiFi Created Apache Hive DDL

hive.ddl CREATE EXTERNAL TABLE IF NOT EXISTS aiyvoice (path STRING, filename STRING, commands0 STRING, ssaddress STRING, receivedFrom STRING, sshost STRING, mimetype STRING, tailfileoriginalpath STRING, uuid STRING, RouteOnAttributeRoute STRING) STORED AS ORC


45637-voicebox.jpg


Resources:

750 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 09:37 AM
Updated by:
 
Contributors
Top Kudoed Authors