Community Articles

TimothySpann · ‎12-26-2017

I wanted to see offer alternatives to running Deep Learning and Machine Learning locally and take advantage of some free hours of cloud time.

It seems that it is trivially easy to integrate calling IBM Watson APIs for all your microservice needs. Apache NiFi makes it super easy and fun.

Using out of the box processors, we can call the InvokeHTTP POST and GET REST APIs of the IBM Watson Platform for Natural Language, Visual Recognition, Personality Analysis, Language Translation and others. These use SSL, so you have to setup a simple StandardSSLContextService in Apache NiFi. Once you know the JVM you are running Apache NiFi with, it's trivial to grab the requirements for that.

By default the password is the very secure, changeit.

A Get REST CALL to NLP

A Post to a Watson REST URL

Some URLs and Calls

curl -X POST --form "images_file=@mypic.jpg" "https://gateway-a.watsonplatform.net/visual-recognition/api/v3/classify?api_key={api-key}&version=2016-05-20"
curl -X POST --form "images_file=@myotherpic.jpg" "https://gateway-a.watsonplatform.net/visual-recognition/api/v3/detect_faces?api_key={api-key}&version=2016-05-20"
curl --user "{username}":"{password}" "https://gateway.watsonplatform.net/natural-language-understanding/api/v1/analyze?version=2017-02-27&text=This+is+a+test&features=sentiment,keywords"
curl --user "{username}":"{password}" "https://gateway.watsonplatform.net/natural-language-understanding/api/v1/analyze?version=2017-02-27&text=Test&features=sentiment,keywords&keywords.sentiment=true"
curl -X POST --user "{username}":"{password}" --header "Content-Type: application/json" --data-binary @{path_to_file}tone.json "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2017-09-21"
curl -X POST --user "{username}":"{password}" --header "Content-Type: application/json" --data-binary @{path_to_file}tone.json "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2017-09-21&sentences=false" curl -X GET --user "{username}":"{password}" "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone?version=2017-09-21
&text=Team%2C%20I%20know%20that%20times%20are%20tough%21%20Product%20sales%20have
%20been%20disappointing%20for%20the%20past%20three%20quarters.%20We%20have%20a%20
competitive%20product%2C%20but%20we%20need%20to%20do%20a%20better%20job%20of%20
selling%20it%21"
curl -X POST --user "{username}":"{password}" --header "Content-Type: application/json" --data-binary @{path_to_file}tone-chat.json "https://gateway.watsonplatform.net/tone-analyzer/api/v3/tone_chat?version=2017-09-21"
Personalality Insights
curl -X POST --user {username}:{password} --header "Content-Type: text/plain;charset=utf-8" --data-binary "@{path_to_file}profile.txt" "https://gateway.watsonplatform.net/personality-insights/api/v3/profile?version=2016-10-20"
Conversation
{
  "url": "https://gateway.watsonplatform.net/conversation/api",
  "username": "user",
  "password": "pass"
}
Discovery
{
  "url": "https://gateway.watsonplatform.net/discovery/api",
  "username": "u",
  "password": "p"
}
curl -X POST -u "{username}":"{password}" -H "Content-Type: application/json" -d '{ "name":"my-first-environment", "description":"exploring environments"}' "api/v1/environments?version=2017-09-01"
Language Translator
{
  "url": "https://gateway.watsonplatform.net/language-translator/api",
  "username": "u",
  "password": "p"
}
curl -X POST --user "{username}":"{password}" --header "Content-Type: application/json" --header "Accept: application/json" --data '{"text":"Hello, world!","source":"en","target":"es"}' "https://gateway.watsonplatform.net/language-translator/api/v2/translate"
Natural Language
{
  "url": "https://gateway.watsonplatform.net/natural-language-understanding/api",
  "username": "u",
  "password": "p"
}
curl --user "{username}":"{password}" "https://gateway.watsonplatform.net/natural-language-understanding/api/v1/analyze?version=2017-02-27&text=SomeText&features=sentiment,keywords"

If you can call it with curl or wget, you can call it with Apache NiFi.

An Overview of Incorporating IBM Watson in Apache NiFi Flows

With Apache NiFi we can subtitute any message you want NLP to analyze. See ${msg} which is expression language.

As you can see just plug your key in there!

IBM Returns some nice clean JSON.

We get our JSON probabilities and can use them as we see fit. The next step I would convert using a schema to Apache AVRO and then to Apache ORC and store in Apache Hive LLAP for queries and analytics.

'

Resources:

Cloudera Community

Community Articles

Integrating IBM Watson Machine Learning APIs with Apache NiFi

Apache NiFi