Creating WordClouds From DataFlows with Apache NiFi and Python

[#1]

Python Word Cloud

Integrating existing Python libraries and scripts is very easy in Apache NiFi. I add the library for both version of Python I have on my system, while moving all new scripts to the 3.x branch.

Install the library for both Python 2.7 and 3.5

pip install wordcloud 
pip3 install wordcloud 

Example Usage

echo "NiFi\nHadoop\nSpark\n" | wordcloud_cli.py --imagefile wordcloud.png

For use in NiFi, I wrap my call with a shell script wc.sh

echo $1 | tr " " "\n" | wordcloud_cli.py 

This will build a PNG for me that I can store in a file system or in HDFS, I updated the filename to add png at the end. This will take a parameter to a shell script (our Tweet) and convert it into words usable for a word cloud. You can use other sources or other methods of splitting words.

I am pulling twitter messages, so I use ReplaceText to replace the flow file with ${msg} which is just the tweet.

Then I execute the Python WordCloud CLI:

Example

References:

• https://amueller.github.io/word_cloud/auto_examples/a_new_hope.html https://amueller.github.io/word_cloud/auto_examples/simple.html#sphx-glr-auto-examples-simple-py https://github.com/amueller/word_cloud