New Contributor
Posts: 1
Registered: ‎05-07-2018

DiskBlockManager throwing exception "Failed to create local dir" when trying to broadcast variable



I am getting a Failed to create local dir in /mnt/tmp/blockmgr ...

This exception is thrown when I am trying to create a broadcast variable(This is on a 6 node cluster). 


This error is not happening consistently. Sometimes it runs fine and some time it fails. I checked the cluster and there is enough disk space left(more than 500GB) and I have permissions to write to /tmp folder. [The exception is not thrown every run].


Any sugestions on this?


Thanks in advance