In MultipleOutput's write(K, V, String) method, you can pass a relative path for the baseOutputPath (last) argument such as "my-sub-dir/my-filename-prefix". This will allow you to produce subdirectories under the job output directory. Please ensure to use only relative paths, and not absolute ones, as otherwise your output paths will not be respected by the FileOutputCommitter process. ... View more

Thank you for following up! Please consider marking the thread resolved so others with similar questions may be able to locate topics with solutions faster. ... View more

Are you ensuring to call multipleOutputs.close() in the cleanup phase? If that is not done, the file won't be closed up, and would be left hanging open for an hour during which you can't see any written data. If you're already doing that, please share your code here so we can take a look at what may be going wrong. ... View more

Thank you - it appears the logs are masking the thread failures at the INFO level. Could you change your logging config into DEBUG for the RM and retry to see if you notice any exceptions pertaining to the handler threads starting up? You can switch it in the log4j.properties file under /etc/hadoop/conf/ on the RM host. I see another email in the cdh-user@ group where something similar was observed for a user on Ubuntu, but that happened to them cause they had a mix-up with the zookeeper package provided by Ubuntu's own repository vs. CDH ones being installed. I am not sure if your Debian repository carries a zookeeper package as well but you should probably check via dpkg/etc. to ensure its a CDH version package and not a Debian-default one. ... View more