Member since
10-22-2018
2
Posts
0
Kudos Received
0
Solutions
10-22-2018
09:24 PM
Hello Every body, I need some help. I'm working on Cloudera Express 5.15.0 When I run a shell action with a Shell script for running spark application, I am constantly confronted with an error. These error seem like same but These are diffrent. At the first time I met this error, There were just two directories 'usercache', 'filecache' under '/DATA01/yarn/nm/usercache/user_01'. And then I remove all directories '/DATA01/yarn/nm/usercache/user_01' on every each Data nodes. Now, I realize that oozie refer different directory from real created directory. Ooze refer : /DATA01/yarn/nm/usercache/user_01/appcache/application_1540195922373_0063/container_1540195922373_0063_01_000003 But Server has : /DATA01/yarn/nm/usercache/user_01/appcache/application_1540195922373_0063/container_1540195922373_0063_01_000002 There is no directory "container_1540195922373_0063_01_000003" on the Data Node. Can I get some help ? Error Log =================================================================================== Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], main() threw exception, Cannot run program "sparkSubmit.sh" (in directory "/DATA01/yarn/nm/usercache/user_01/appcache/application_1540195922373_0063/container_1540195922373_0063_01_000003"): error=2, No such file or directory java.io.IOException: Cannot run program "sparkSubmit.sh" (in directory "/DATA01/yarn/nm/usercache/user_01/appcache/application_1540195922373_0063/container_1540195922373_0063_01_000003"): error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at org.apache.oozie.action.hadoop.ShellMain.execute(ShellMain.java:111) at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:70) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:81) at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:59) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:235) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1920) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:247) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 17 more
... View more
Labels:
- Labels:
-
Apache Oozie
07-11-2018
01:44 AM
In my case, even I set the values of "hive.llap.io.threadpool.size" and "hive.llap.daemon.num.executors" same. Still I got the error like as given below.. "2018-07-10 20:56:26,421 - LLAP app 'llap0' in 'RUNNING_PARTIAL' state. Live Instances : '2'. Desired Instances : '3' after 619.594609976 secs." When I changed "Number of Node(s) for running Hive LLAP daemon (num_llap_nodes_for_llap_daemons) " to 2. It's working well.
... View more