Cloudera Community
dcote
user rank
Explorer
My Accepted Solutions
Show All

Re: Spark's faill durring persist()

by Explorer in Support Questions
‎07-15-2015 05:50 AM
‎07-15-2015 05:50 AM
t's possible that you are overwhelming the CPU on the hosts by using StorageLevel.MEMORY_AND_DISK_SER as this is a CPU intensive storage strategy: https://spark.apache.org/docs/1.3.0/programming-guide.html#rdd-persistence Are you able to use deserialized objects instead? Using StorageLevel.MEMORY_AND_DISK will be less CPU intensive. ... View more

Re: How to run Oozie workfllow or action as anothe...

by Explorer in Support Questions
‎05-15-2015 12:39 PM
‎05-15-2015 12:39 PM
Shell actions are not allowed to run as another user as sudo is blocked. If you want a yarn application to run as someone other than yarn (i.e. the submitter), then you'd want to run in a secured environment so that the containers are started up by the submitting user. See Shell Action Caveats here: http://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/ Hope this helps ... View more

Re: How to run Oozie workfllow or action as anothe...

by Explorer in Support Questions
‎05-08-2015 11:49 AM
‎05-08-2015 11:49 AM
Hi Martin,   I believe Hue runs Oozie workflows as the user you are logged into Hue as.  The easiest way to run a workflow as a different user is to log into Hue as a different user.  Per the Oozie documentation, there are actions where you are not allowed to override the user/group:   https://oozie.apache.org/docs/3.2.0-incubating/WorkflowFunctionalSpec.html#a6_User_Propagation   ... View more

Re: CDH 5.3 CLASSPATH Issue?

by Explorer in Support Questions
‎01-21-2015 05:29 AM
‎01-21-2015 05:29 AM
The workaround for this is to break the link in your CDH bundle on the gateway machines where you submit the mapreduce jobs at this location -->$CDH_ROOT/lib/hadoop-mapreduce   The link was put there in error in CDH 5.3 and a patch has been submitted to get it removed in the next release.  Searching all directories that show up when running the command 'hadoop classpath' for the joda-time-1.6.jar will help you find the errant link if you use non-default locations for the install.  The fix should apply to spark and MR jobs. ... View more
Contact Me
Online
Offline
Last Visited
‎07-22-2015 11:56 AM
Community Statistics
Member Since ‎09-22-2014 10:40 AM
Last Visited ‎07-22-2015 11:56 AM
Posts 5
Kudos received 1