- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
Created on 03-13-2017 12:58 PM
Problem
Here is my user I run %livy.sparkr with
%sh whoami admin
Permissions set to directory and file:
%sh ls -lrt /home/admin total 12 -rwxr-xr-x 1 admin admin 3950 Mar 9 13:12 common.r drwxrwxr-x 2 admin admin 4096 Mar 10 15:25 analysis drwxrwxr-x 3 admin admin 4096 Mar 13 12:49 motorAnalysis %sh ls -lrt /home/admin/motorAnalysis total 8 -rwxrwxr-x 1 admin admin 3950 Mar 10 16:49 common.r drwxrwxr-x 5 admin admin 4096 Mar 10 16:49 data
Running %livy.sparkr I am getting
%livy.sparkr ... source("/home/admin/motorAnalysis/common.r") null device 1 Error in file(filename, "r", encoding = encoding) : cannot open the connection Calls: source -> file In addition: Warning message: In file(filename, "r", encoding = encoding) : cannot open file '/home/admin/motorAnalysis/common.r': No such file or directory
Solution
Restart livy interpreter from Zeppelin UI fixes the problem
Created on 03-13-2017 08:46 PM
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
Daniel Kozlowski, Any explanation on why was this error happening on 1st attempt ? And how did it get resolved on interpreter restart ?
Created on 03-14-2017 06:47 AM
- Mark as Read
- Mark as New
- Bookmark
- Permalink
- Report Inappropriate Content
This problem happened at the customer's site on HDP 2.5.0. The customer had already spark job running on RM UI. Then created a folder, permissions, etc using sh interpreter or command line. Running livy.sparkr using source function displayed "No such file or directory" even though the file was there. Restarting livy interpreter fixed the issue.
BTW: I have tested it out on my local HDP 2.5.3 and all works fine - no livy interpreter needs to be restarted.