Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Super Guru

Apache NiFi Processor for Apache MXNet SSD: Single Shot MultiBox Object Detector (Deep Learning)

The news is out, Apache MXNet has added a Java API. So as soon as I could I got my hands on the maven repo and an example program and got to work writing a new Apache NiFi processor for it.

I have run this on standalone Apache NiFi 1.8.0 and on HDF 3.3 - Apache NiFi 1.8.0 and both work. So anyone who wants to be an alpha tester, please download it and give it a try.

Apache MXNet SSD is a good example of a pretrained deep learning model that works pretty well for general images in a use cases especially around people and cars. You can fine-tune this with some more images and runs: https://mxnet.incubator.apache.org/faq/finetune.html

The nice thing is now we can start including Apache MXNet as part of Java applications such as Kafka Streams, Apache Storm, Apache Spark, Spring Boot and other use cases using Java. I could potentially inject this into a Hive UDF (https://community.hortonworks.com/articles/39980/creating-a-hive-udf-in-java.html#comment-40026) or Pig UDF. The performance may be fast enough. We now have four Java options for Deep Learning: DL4J, H2O, Tensorflow and Apache MXNet. Unfortunately, both TensorFlow and MXNet Java APIs are not quite production ready.

I may do some further research on running MXNet as a Hive UDF, it would be cool to have in a query.

For those who don't want to setup a development environment with JDK 8+, Maven 3.3+ and git, you can download a pre-built nar file here: https://github.com/tspannhw/nifi-mxnetinference-processor/releases/tag/v1.0.

As part of the recent release of HDF 3.3, I have upgraded my OpenStack Centos 7 cluster.

94459-hdf33.png

Important Caveats
Notice, the Java API is in preview and so is this processor. Do not use this in production! This is in development and I am the only one working on it. The Java API from Apache MXNet is in flux and will be changing. See the POM as it is tied to the OSX/Mac version of the library. You will need to change that. You will need to download the pre-built MXNet model and place it in a directory accessible to Apache NiFi server/cluster. I am still cleaning up the rectangle code for identifying objects in the pictures.

94457-2a681d2c-b8fb-43cb-9961-7ff001726951.jpg

As you will notice, my rectangle drawing is a bit off. I need to work on that.

Once you drop your built nar file and models in the nifi/lib directory and restart Apache NiFi, you can add it to your canvas.

94458-addyourprocessor.png

We need to feed it some images. You can use my web cam processor, an image URL feed or local files.

94462-grabsomeimages.png

To grab images from an HTTPS site, you need an SSL Context Service like this StandardSSLContextService below. You will need to point to the cacerts used by the JRE/JDK running your Apache NiFi node. The default password in Java is changeme. Hopefully you have changed it.

94461-httpsconfig.png

To configure my new processor, just put in the full path to the model directory and then "/resnet50_ssd_model" as that is the prefix for the model.

94463-configurelabel.png

Our example flow with new processor being fed by traffic cameras, webcams, local files and local webcam.

94465-mxnetflow.png

Some output of our flow:

94464-flowfileattributes.png

Our top 5 probabilities and labels

94456-mxnet-prob.png

Example Data:

{
  "ymin_1" : "456.01",
  "ymin_5" : "159.29",
  "ymin_4" : "235.83",
  "ymin_3" : "206.64",
  "ymin_2" : "383.84",
  "label_5" : "person",
  "xmax_5" : "121.14",
  "label_4" : "bicycle",
  "xmax_4" : "137.89",
  "label_3" : "dog",
  "xmax_3" : "179.14",
  "ymax_1" : "150.66",
  "ymax_2" : "418.95",
  "ymax_3" : "476.79",
  "label_2" : "bicycle",
  "label_1" : "car",
  "probability_4" : "0.22",
  "probability_5" : "0.13",
  "probability_2" : "0.90",
  "xmin_5" : "88.93",
  "probability_3" : "0.82",
  "ymax_4" : "413.43",
  "probability_1" : "1.00",
  "ymax_5" : "190.04",
  "xmax_2" : "149.96",
  "xmax_1" : "72.03",
  "xmin_3" : "83.82",
  "xmin_4" : "93.05",
  "xmin_1" : "312.21",
  "xmin_2" : "155.96"
}

Resources:

Source:

https://github.com/tspannhw/nifi-mxnetinference-processor

Video walk-through: https://www.youtube.com/watch?v=Q4dSGPvqXSA&t=196s&list=PL-7XqvSmQqfTSihuoIP_ZAnN7mFIHkZ_e&index=17

mxnet-processor.xml

Download the artifacts listed: https://github.com/apache/incubator-mxnet/tree/java-api/scala-package/examples/src/main/java/org/apa...

Maven POM (I used Java 8 and Maven 3.3.9)

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements. See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License. You may obtain a copy of the License at
  http://www.apache.org/licenses/LICENSE-2.0
  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>


    <parent>
        <groupId>com.dataflowdeveloper.mxnet</groupId>
        <artifactId>inference</artifactId>
        <version>1.0</version>
    </parent>


    <artifactId>nifi-mxnetinference-processors</artifactId>
    <packaging>jar</packaging>


    <dependencies>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
            <version>1.8.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-mock</artifactId>
            <version>1.8.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>


        <dependency>
            <groupId>org.apache.mxnet</groupId>
            <artifactId>mxnet-full_2.11-osx-x86_64-cpu</artifactId>
            <version>1.3.1-SNAPSHOT</version>
        </dependency>


    </dependencies>
</project>

I have moved from Eclipse to IntelliJ from my builds. I am looking at Apache Netbeans as well.

94460-unittest.png

488 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 05:30 AM
Updated by:
 
Contributors
Top Kudoed Authors