As part of a live Drone ingest, I want to identify what in the image. The metadata provides a ton of information on GPS, altitude and image characteristics, but not what's in the image.
IBM, Microsoft and Google all have APIs that do a good job of this and they for the most part of "free" tiers. I wanted to run something locally using libraries installed on my cluster. For my first option, I used TensorFlow Inception-v3 Image Recognition. In future articles I will cover PaddlePaddle, OpenCV and some other Deep Learning and non-deep learning options for Image Recognition. Also I will show the entire Drone to Front-End flow including Phoenix, Spring Boot, Zeppelin, LeafletJS and more. This will be done as part of a meetup presentation with a certified drone pilot.
To Run My TensorFlow Binary From HDF 2.0 I use the ExecuteStreamCommand to run a shell script containing the information below:
source /usr/local/lib/bazel/bin/bazel-complete.bash export JAVA_HOME=/opt/jdk1.8.0_101/ hdfs dfs -get /drone/raw/$@ /tmp/ /opt/demo/tensorflow/bazel-bin/tensorflow/examples/label_image/label_image --image="/tmp/$@" --output_layer="softmax:0" --input_layer="Mul:0" --input_std=128 --input_mean=128 --graph=/opt/demo/tensorflow/tensorflow/examples/label_image/data/tensorflow_inception_graph.pb --labels=/opt/demo/tensorflow/tensorflow/examples/label_image/data/imagenet_comp_graph_label_strings.txt 2>&1| cut -c48-
In my script I pull the file out of HDFS (that was loaded by HDF 2.0) and then run the binary versino of TensorFlow that I compiled for Centos7. If you can't or don't want to install Bezel and build that, they you can run the python script, it's a little bit slower and has slightly different output.
python /usr/lib/python2.7/site-packages/tensorflow/models/image/imagenet/classify_image.py --image_file
The C++ version that I compiled is Google's example and you can take a look at it: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/label_image/README.md https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/label_image/main.cc It's very clean code if you wish to tweak it.
Installing TensorFlow
You must have JDK 1.8 (and know the path), not just JRE 1.8. Also you need Python 2.7 or 3.+ and PIP. You need to install Google's Bezel build tool.
sudo yum groupinstall "Development Tools" sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-develsudo yum -y install epel_release sudo yum -y install gcc gcc-c++ python-pip python-devel atlas atlas-devel gcc-gfortran openssl-devel libffi-devel pip install --upgrade numpy scipy wheel cryptography export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.10.0-cp27-none-linux_x86_64.whl git clone https://github.com/tensorflow/tensorflow cd tensorflow/ wget https://storage.googleapis.com/download.tensorflow.org/models/inception_dec_2015.zip -O tensorflow/examples/label_image/data/inception_dec_2015.zip unzip tensorflow/examples/label_image/data/inception_dec_2015.zip -d tensorflow/examples/label_image/data/ cd tensorflow/examples/label_image /opt/demo/bazel/output/bazel label_image
Run Results of TensorFlow
python classify_image.py --image_file /opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg solar dish, solar collector, solar furnace (score = 0.98316) window screen (score = 0.00196) manhole cover (score = 0.00070) radiator (score = 0.00041) doormat, welcome mat (score = 0.00041) bazel-bin/tensorflow/examples/label_image/label_image --image=/opt/demo/dronedataold/Bebop2_20160920083655-0400.jpg solar dish (577): 0.983162I window screen (912): 0.00196204I manhole cover (763): 0.000704005I radiator (571): 0.000408321I doormat (972): 0.000406186
The image is a picture of solar panels on a residential black tar roof.
Resources
- https://www.tensorflow.org/versions/r0.10/tutorials/index.html
- https://www.tensorflow.org/versions/r0.10/tutorials/image_recognition/index.html
- http://hoolihan.net/blog-tim/2016/03/02/installing-tensorflow-on-centos/
- https://databricks.com/blog/2016/01/25/deep-learning-with-apache-spark-and-tensorflow.html
- http://neuralnetworksanddeeplearning.com/chap1.html http://colah.github.io/posts/2014-07-Conv-Nets-Modular/
- http://neuralnetworksanddeeplearning.com/chap6.html
- https://www.tensorflow.org/versions/r0.10/tutorials/deep_cnn/index.html
- https://www.tensorflow.org/versions/r0.10/tutorials/mnist/beginners/index.html
- http://image-net.org/
- https://www.bazel.io/versions/master/docs/install.html
- https://github.com/bazelbuild/bazel/releases
- http://tecadmin.net/install-java-8-on-centos-rhel-and-fedora/
- https://cloud.google.com/vision/
- https://www.microsoft.com/cognitive-services/en-us/computer-vision-api
- http://www.ibm.com/watson/developercloud/visual-recognition/api/v3/#introduction
