Support Questions

Find answers, ask questions, and share your expertise

Deep Learning on HDP

Super Guru

Has anyone gotten TensorFlow or DeepLearning4J running on top of HDP 2.4?

I am interested in the setup.


@Timothy Spann see webinar @Dhruv Kumar ran last year using older version of HDP


@Timothy Spann If it's still relevant, there is a project called TensorFrames that is pretty much a wrapper for Spark's DataFrames: Although I am not sure if it is being actively developed, last time I checked it looked functional.

New Contributor

Hi Tim - I know this is an old question and we've chatted offline. I just wanted to have it on the record here that Deeplearning4j is just a Hadoop job that can be submitted as a JAR file. It's that simple.

Super Guru

Sweet. This will also work with HDP 2.5.

Rising Star


I have the same exact question but I couldn't link or associate listed variations of deep learning applications with the use of HDP.

How am I supposed to run any of them on my Hadoop cluster? In which practical, infrastructural way do DL applications and Hadoop applications get related?

Thanks in advance...

Super Guru

in HDP 3.0, YARN 3 will run TensorFlow jobs in docker containers.

Tensorflow on Spark

Caffe on Spark

both run on Spark on YARN and will be distributed in your Hadoop cluster

For many applications I am running tensorflow python scripts (already trainined) on NiFi nodes, but could run on HDP nodes which all have python installed.

Other option is to run TensorFlow with TensorFlow Serving and call it from existing applications, NiFi and big data apps via gRPC

Super Guru