Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st. We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here. Need help or have questions? Drop us a line at [email protected]
%sh
ls -lrt /home/admin
total 12
-rwxr-xr-x 1 admin admin 3950 Mar 9 13:12 common.r
drwxrwxr-x 2 admin admin 4096 Mar 10 15:25 analysis
drwxrwxr-x 3 admin admin 4096 Mar 13 12:49 motorAnalysis
%sh
ls -lrt /home/admin/motorAnalysis
total 8
-rwxrwxr-x 1 admin admin 3950 Mar 10 16:49 common.r
drwxrwxr-x 5 admin admin 4096 Mar 10 16:49 data
Running %livy.sparkr I am getting
%livy.sparkr
...
source("/home/admin/motorAnalysis/common.r")
null device
1 Error in file(filename, "r", encoding = encoding) :
cannot open the connection
Calls: source -> file
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
cannot open file '/home/admin/motorAnalysis/common.r': No such file or directory
Solution
Restart livy interpreter from Zeppelin UI fixes the problem
This problem happened at the customer's site on HDP 2.5.0. The customer had already spark job running on RM UI. Then created a folder, permissions, etc using sh interpreter or command line. Running livy.sparkr using source function displayed "No such file or directory" even though the file was there. Restarting livy interpreter fixed the issue.
BTW: I have tested it out on my local HDP 2.5.3 and all works fine - no livy interpreter needs to be restarted.