Community Articles

TimothySpann · ‎03-02-2018

62526-56690-icon.png

62527-56692-dwsberling.jpg

This is for people preparing to attend my talk on Deep Learning at DataWorks Summit Berling 2018 (https://dataworkssummit.com/berlin-2018/#agenda) on Thursday April 19, 2018 at 11:50AM Berlin time.

Another way to work with Apache MXNet is by using your Apache Zeppelin notebook to run your Python deep learning scripts.

Apache Zeppelin Notebook

62622-mxnetinzeppelintable.png

As you can see we can format that data as a table using Apache Zeppelin display technology.

Use this print statement:

print("%table top1pct\ttop1\top2\ttop2pct\ttop3pct\ttop3\ttop4pct\ttop4\ttop5pct\ttop5\timagefilename\truntime\tuuid\n" + top1pct + "\t" + top1 + "\t" +  top2pct + "\t" + top2 + "\t" +  top3pct + "\t" + top3  + "\t" + top4pct + "\t" + top4 + "\t" +  top5pct + "\t" + top5  + "\t" + filename  + "\t" + str(round(end - start)) + "\t" + uniqueid + "\n" )

We use the pyspark interpreter to run this Python script, but there's no Spark in here yet.

62623-mxnetinzeppelintop.png

This data also gets loaded in Apache Hive via Apache NiFi as shown here:

62624-datainhive2ui.png

Deep Learning Models

You will need to download the pre-built Inception models and reference them on your server.

synset.txt

Inception-BN-0000.params

Inception-BN-symbol.json

See: https://mxnet.incubator.apache.org/tutorials/embedded/wine_detector.html

curl --header 'Host: data.mxnet.io' --header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Firefox/45.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-US,en;q=0.5' --header 'Referer: http://data.mxnet.io/models/imagenet/' --header 'Connection: keep-alive' 'http://data.mxnet.io/models/imagenet/inception-bn.tar.gz' -o 'inception-bn.tar.gz' -L

curl http://data.mxnet.io/models/imagenet/synset.txt

More Models

http://data.mxnet.io/models/imagenet/

Source Code

https://github.com/tspannhw/mxnet-in-notebooks

https://github.com/tspannhw/ApacheBigData101

References

If you want to run in DSX or Jupyter:

https://community.hortonworks.com/articles/176784/deep-learning-101-using-apache-mxnet-in-dsx-notebo...

Setup

If you need to setup Apache MXNet on HDF: https://community.hortonworks.com/articles/174227/apache-deep-learning-101-using-apache-mxnet-on-an....

Other Articles in The Series