I am getting a java.io.IOException: Failed to create local dir in /mnt/tmp/blockmgr ...
This exception is thrown when I am trying to create a broadcast variable(This is on a 6 node cluster).
This error is not happening consistently. Sometimes it runs fine and some time it fails. I checked the cluster and there is enough disk space left(more than 500GB) and I have permissions to write to /tmp folder. [The exception is not thrown every run].
Any sugestions on this?
Thanks in advance