Integration Apache OpenNLP 1.8.4 into Apache NiFi 1.5 For Real-Time Natural Language Processing of Live Data Streams
This is an update to the existing processor. This one seems to work better and faster.
Versions
Apache OpenNLP 1.8.4 with Name, Location and Date Processing.
I also improved the output format and added Date parsing.
Example Output
nlp_location_1
China
nlp_name_1
Andrew Turner
Release
https://github.com/tspannhw/nifi-nlp-processor/releases/tag/2.0
Installation
- Download NAR here: https://github.com/tspannhw/nifi-nlp-processor/releases/tag/2.0
- Install nar file to /usr/hdf/current/nifi/lib/
- Create a model directory with permissions for nifi user
- Download models (see below)
- Restart Apache NiFi via Ambari
Download Models
wget http://opennlp.sourceforge.net/models-1.5/en-ner-date.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-location.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-money.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-organization.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-percentage.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-person.bin wget http://opennlp.sourceforge.net/models-1.5/en-ner-time.bin wget http://opennlp.sourceforge.net/models-1.5/en-chunker.bin wget http://opennlp.sourceforge.net/models-1.5/en-parser-chunking.bin wget http://opennlp.sourceforge.net/models-1.5/en-token.bin wget http://opennlp.sourceforge.net/models-1.5/en-sent.bin wget http://opennlp.sourceforge.net/models-1.5/en-pos-maxent.bin wget http://opennlp.sourceforge.net/models-1.5/en-pos-perceptron.bin
Resources:
- https://community.hortonworks.com/articles/76240/using-opennlp-for-identifying-names-from-text.html
- https://community.hortonworks.com/articles/163776/parsing-any-document-with-apache-nifi-15-with-apac...
- https://community.hortonworks.com/articles/76924/data-processing-pipeline-parsing-pdfs-and-identify....
- https://community.hortonworks.com/articles/80418/open-nlp-example-apache-nifi-processor.html
- https://community.hortonworks.com/articles/76935/using-sentiment-analysis-and-nlp-tools-with-hdp-25....
- https://community.hortonworks.com/articles/142686/real-time-ingesting-and-transforming-sensor-and-so...
