Cloudera Community
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
nbest
user rank
Explorer

Oozie shell action environment and .impalarc

by Explorer in Support Questions
‎10-13-2015 09:27 AM
‎10-13-2015 09:27 AM
I have an Oozie shell action that calls impala-shell and has been working fine.  I wanted to set some defaults in ~/.impalarc which works fine from the shell but doesn't seem to get applied during workflow execution.  When I look at the environment variable values in the shell action log it says:   . . . USER=etl-user: . . . HOME=/home/: . . .   USER looks right but I expect HOME to be either /home/$USER or /home/etl-user.   Does this influence where impala-shell looks for .impalarc when running under an Oozie shell action?   I was hoping to get this done by distributing my .impalarc to ~etl-user on all of the cluster nodes and not chenge the workflow definition but no joy.   The objective is to change the default --impalad=host:port argument to point at the Impala high availabilty proxy that we have set up. ... View more
Labels:

Re: Issue with: Hive Action - Oozie

by Explorer in Support Questions
‎09-03-2015 01:46 PM
‎09-03-2015 01:46 PM
From where should I copy hive-site.xml?  I tried a couple that I found under /opt/cloudera/parcels individually but no joy. ... View more

Re: spark: Exception in thread "main" java.lang.No...

by Explorer in Support Questions
‎04-28-2015 01:22 PM
‎04-28-2015 01:22 PM
Two solutions that I reasoned out by staring at the various scripts in /usr/lib/spark/bin   SPARK_CLASSPATH=/usr/share/java/slf4j-simple.jar spark-shell --master local   spark-shell --driver-class-path /usr/share/java/slf4j-simple.jar --master local   The first approach is deprecated according to the output. ... View more

Re: spark: Exception in thread "main" java.lang.No...

by Explorer in Support Questions
‎04-28-2015 01:06 PM
‎04-28-2015 01:06 PM
I have a similar problem trying to run spark-shell from CDH 5.4 under 14.04. I was able to get around this by saying:   SPARK_PRINT_LAUNCH_COMMAND=true spark-shell   :to get the underlying java invocation and then tack an SLF4J jar onto the classpath there:   /usr/lib/jvm/java-7-oracle/bin/java -cp :/usr/lib/spark/conf:/usr/lib/spark/lib/spark-assembly-1.3.0-cdh5.4.0-hadoop2.6.0-cdh5.4.0.jar:/etc/hadoop/conf::/usr/lib/spark/lib/spark-assembly.jar::/usr/lib/hadoop/lib/*:/usr/lib/hadoop/*:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/*:/usr/lib/hive/lib/*:/usr/lib/flume-ng/lib/*:/usr/lib/paquet/lib/*:/usr/lib/avro/lib/*:/usr/share/java/slf4j-simple.jar -XX:MaxPermSize=128m -Dscala.usejavacp=true -Xms512m -Xmx512m org.apache.spark.deploy.SparkSubmit --class org.apache.spark.repl.Main spark-shell   There must be something missing from one of the scripts or JARs in the CDH Ubuntu repo.  Any suggestions on how to hack it so that spark-shell works until this is addressed?   ... View more
Contact Me
Online
Offline
Last Visited
‎04-20-2016 03:42 PM
Community Statistics
Member Since ‎04-28-2015 12:53 PM
Last Visited ‎04-20-2016 03:42 PM
Posts 7