Hi, I’m running spark on k8s and would love to access files that are submitted with —files on the driver (not executor). I know files are uploaded to /tmp/spark-XXXX/ folder that is random, but I can’t find any environment variable that I can use in driver to access them.
the reason I want this is to pass a config file in —files with some parameters , and I want to access this file before I initialize spark context.