%sh
ls -lrt /home/admin
total 12
-rwxr-xr-x 1 admin admin 3950 Mar 9 13:12 common.r
drwxrwxr-x 2 admin admin 4096 Mar 10 15:25 analysis
drwxrwxr-x 3 admin admin 4096 Mar 13 12:49 motorAnalysis
%sh
ls -lrt /home/admin/motorAnalysis
total 8
-rwxrwxr-x 1 admin admin 3950 Mar 10 16:49 common.r
drwxrwxr-x 5 admin admin 4096 Mar 10 16:49 data
Running %livy.sparkr I am getting
%livy.sparkr
...
source("/home/admin/motorAnalysis/common.r")
null device
1 Error in file(filename, "r", encoding = encoding) :
cannot open the connection
Calls: source -> file
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
cannot open file '/home/admin/motorAnalysis/common.r': No such file or directory
Solution
Restart livy interpreter from Zeppelin UI fixes the problem
This problem happened at the customer's site on HDP 2.5.0. The customer had already spark job running on RM UI. Then created a folder, permissions, etc using sh interpreter or command line. Running livy.sparkr using source function displayed "No such file or directory" even though the file was there. Restarting livy interpreter fixed the issue.
BTW: I have tested it out on my local HDP 2.5.3 and all works fine - no livy interpreter needs to be restarted.