Hi All, I am working on a spark java wrapper which uses third party libraries, which will read files from a hard coded directory name say "resdata" from where job executes. I know this is twisted but will try to explain. when I execute the job it is trying to find the required files in the path something like this below, /data<xx>/Hadoop/yarn/local/<userspecificpath>/appcache/application_xxxxx_xxx/container_00_xxxxx_xxx/resdata I am assuming it is looking for the files in the current data directory , under that looking for directory name "resdata". At this point I don't know how to configure the current directory to any path on hdfs or local. So looking for options to create directory structure similar to what the third party libraries expecting and copying required files over there. This I need to do on each node. I am working on spark 2.2.0 Please help me in achieving this?
... View more
