Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Master Guru

Integration Apache OpenNLP 1.8.4 into Apache NiFi 1.5 For Real-Time Natural Language Processing of Live Data Streams


This is an update to the existing processor. This one seems to work better and faster.

Versions

Apache OpenNLP 1.8.4 with Name, Location and Date Processing.

I also improved the output format and added Date parsing.

64804-nlpcode.png

64805-nlpjunit.png





64819-nlpflow.png


Example Output

nlp_location_1 China

nlp_name_1 Andrew Turner


Release

https://github.com/tspannhw/nifi-nlp-processor/releases/tag/2.0

Installation

  1. Download NAR here: https://github.com/tspannhw/nifi-nlp-processor/releases/tag/2.0
  2. Install nar file to /usr/hdf/current/nifi/lib/
  3. Create a model directory with permissions for nifi user
  4. Download models (see below)
  5. Restart Apache NiFi via Ambari

Download Models

wget http://opennlp.sourceforge.net/models-1.5/en-ner-date.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-location.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-money.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-organization.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-percentage.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-person.bin

wget http://opennlp.sourceforge.net/models-1.5/en-ner-time.bin

wget http://opennlp.sourceforge.net/models-1.5/en-chunker.bin

wget http://opennlp.sourceforge.net/models-1.5/en-parser-chunking.bin

wget http://opennlp.sourceforge.net/models-1.5/en-token.bin

wget http://opennlp.sourceforge.net/models-1.5/en-sent.bin

wget http://opennlp.sourceforge.net/models-1.5/en-pos-maxent.bin

wget http://opennlp.sourceforge.net/models-1.5/en-pos-perceptron.bin

Resources:

1,236 Views