If that doesn't work, try running as the HDFS superuser sudo -u hdfs hdfs dfsadmin -safemode leave sudo -u hdfs hdfs dfs -mkdir /user/admin sudo -u hdfs hdfs dfs -chown root:hdfs /user/admin ... View more

I have the same error while running Spark Streaming with flume using python: ERROR LiveListenerBus: Listener EventLoggingListener threw an exception java.io.IOException: All datanodes DatanodeInfoWithStorage[172.17.0.2:50010,DS-8a8dcd2e-a60e-44af-ae75-4679b6341e4a,DISK] are bad. Aborting... at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1227) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.processDatanodeError(DFSOutputStream.java:999) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:506) ... View more

Hi Smanjee, I think you missed the right link here: "This is well documented here so no reason for me to regurgitate that." ... View more