Support Questions
Find answers, ask questions, and share your expertise

How cloudera manager execute parcel_env.sh

How cloudera manager execute parcel_env.sh

Contributor

Hi,

 

I have remotely setup a custom parcel and successfully activated . I am curious to know how and when cloudera manager execute the ../meta/parcel_env.sh file. In my case I tried to set some env variable in parcel_env.sh . but I could not find them . Please help.

 

Kind Regards

Khirod

10 REPLIES 10

Re: How cloudera manager execute parcel_env.sh

Please have a read of this page which explains how the environment
variables can be used. They are for use within the parcel and won't be
available in the general login shell of any user.

https://github.com/cloudera/cm_ext/wiki/The%20parcel%20defines%20script

Regards,
Gautam Gopalakrishnan

Re: How cloudera manager execute parcel_env.sh

Contributor

Hi GautamG,

 

I have already gone through the given link. I just curios to know , is there any way to validate if PATH set done properly or not? before I got a ClassNotFoundException?

 

 

Regards

Khirod

Re: How cloudera manager execute parcel_env.sh

Sorry I am unsure what that means. PATH refers to executables and
ClassNotFound is from setting the classpath. Did you have an issue with
logs/screenshots you can share?

Regards,
Gautam Gopalakrishnan

Re: How cloudera manager execute parcel_env.sh

Contributor

In  parcel.json  we have a scripts section, where I have defined all required class paths . My question was is this script called by cloudera manager at the time of distributing parcels ? Or is there any other way/process to call this script? Not sure why !  In my case the defined script never executed by cloudera manager when deploying my custom parcel.

 

 

 

 

 

"scripts": {
    "defines": "myparcel_env.sh"
  },


 

Regards

Khirod

 

 

Re: How cloudera manager execute parcel_env.sh

Contributor

Just curious to know if the parcel_env.sh is executed by shell or it just used to source the file to set up the required environment variables?

 

My requirement is I have some config files and some jar file which need to be configure and place in proper path. Can I guess if I can do it through a Oozie job executor, not sure how about this idea. If something possible in parcel distribution only would be great.

 

Regards

Khirod

Re: How cloudera manager execute parcel_env.sh

The parcel_env.sh is not used outside of the parcel. If you want something
set up within your Oozie job, it should not depend on this script

Regards,
Gautam Gopalakrishnan

Re: How cloudera manager execute parcel_env.sh

Contributor

That I thought , oozie job may fulfills my requirement . Will update here how it goes !!

 

Regards

Khirod

Re: How cloudera manager execute parcel_env.sh

Contributor

I tried to execute shell script through an oozie job , It seems it just executed in jobTracker host not in other nodes. I am expecting the script to be executed in all the nodes. did I required any other specific configuration . Or I missed anything here.

 

workflow.xml

------------------------------

 

<workflow-app name="script_oozie_job" xmlns="uri:oozie:workflow:0.3">
<start to='Test' />
<action name="Test">
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>CopyFiles.sh</exec>
<argument>${argument1}</argument>
<file>hdfs://nameNode-host:8020/user/oozie/script/script_oozie_job/CopyFiles.sh#CopyFiles.sh</file>
</shell>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Script failed</message>
</kill>
<end name='end' />

 

 

job.properties

---------------------------------

 

nameNode=hdfs://nameNode-host:8020
jobTracker=jobTacker-host:8032
queueName=default
argument1=""
oozie.wf.application.path=hdfs://nameNode-host:8020/user/oozie/script/script_oozie_job

 

 

 

Regards

-Khirod

Re: How cloudera manager execute parcel_env.sh

Could I request that you please start a new thread for the Oozie question. We like to keep one thread per question to keep it clear.

The forum for Oozie is
http://community.cloudera.com/t5/Batch-Processing-and-Workflow/bd-p/JavaAPI
Regards,
Gautam Gopalakrishnan