This is for people preparing to attend my talk on Deep Learning at DataWorks Summit Berling 2018 (https://dataworkssummit.com/berlin-2018/#agenda) on Thursday April 19, 2018 at 11:50AM Berlin time.
Another way to work with Apache MXNet is by using your Apache Zeppelin notebook to run your Python deep learning scripts.
Apache Zeppelin Notebook
As you can see we can format that data as a table using Apache Zeppelin display technology.
Use this print statement:
print("%table top1pct\ttop1\top2\ttop2pct\ttop3pct\ttop3\ttop4pct\ttop4\ttop5pct\ttop5\timagefilename\truntime\tuuid\n" + top1pct + "\t" + top1 + "\t" + top2pct + "\t" + top2 + "\t" + top3pct + "\t" + top3 + "\t" + top4pct + "\t" + top4 + "\t" + top5pct + "\t" + top5 + "\t" + filename + "\t" + str(round(end - start)) + "\t" + uniqueid + "\n" )We use the pyspark interpreter to run this Python script, but there's no Spark in here yet.
This data also gets loaded in Apache Hive via Apache NiFi as shown here:
Deep Learning Models
You will need to download the pre-built Inception models and reference them on your server.
synset.txt
Inception-BN-0000.params
Inception-BN-symbol.json
See: https://mxnet.incubator.apache.org/tutorials/embedded/wine_detector.html
curl --header 'Host: data.mxnet.io' --header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Firefox/45.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --header 'Accept-Language: en-US,en;q=0.5' --header 'Referer: http://data.mxnet.io/models/imagenet/' --header 'Connection: keep-alive' 'http://data.mxnet.io/models/imagenet/inception-bn.tar.gz' -o 'inception-bn.tar.gz' -L
curl http://data.mxnet.io/models/imagenet/synset.txt
More Models
http://data.mxnet.io/models/imagenet/
Source Code
https://github.com/tspannhw/mxnet-in-notebooks
https://github.com/tspannhw/ApacheBigData101
References
If you want to run in DSX or Jupyter:
Setup
If you need to setup Apache MXNet on HDF: https://community.hortonworks.com/articles/174227/apache-deep-learning-101-using-apache-mxnet-on-an....
Other Articles in The Series
https://community.hortonworks.com/articles/155435/using-the-new-mxnet-model-server.html
