Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Why my parcel can't be sourced or run?

avatar
Contributor

My environment script has never been sourced or run even though my parcel is at actived status for each host. The parcel files were indeed distributed into each host.

I also tried restart the cluster, it was not working. Then I tried to manually source the script in a host, it could work, so the script itself has no problem.

I am wondering if any procedure was missed? Or any naming rule for the script? How can i tell if my parcel's shell script has been sourced by CM?

1 ACCEPTED SOLUTION

avatar
Contributor

https://github.com/cloudera/cm_ext/wiki/The%20parcel.json%20file#providing-tags

 

As it discusses, if you want your parcel to affect an existing service, it should provide the 'foo-plugin' tag for that service, or cdh-plugin if you really want to affect all of them.

 

Only one parcel can provide the required tag ('cdh' in this case), and obviously the CDH parcel provides that.

 

What is your parcel providing that you want it to affect all services? LZO is a highly unusual example in this regard.

View solution in original post

12 REPLIES 12

avatar
Contributor

The environment script is sourced as part of starting processes - it's not a global thing that happens when you activate the parcel. So, assuming you start a process after activating the parcel, and assuming your parcel's 'provides' tags match what the process' service is looking for, it should get sourced.

 

avatar
Contributor
He, philipl, thanks for your response! I thought after activating the parcel, the source will be automatically sourced.
Could you tell me how to "start a process" from CM browser?

avatar
Contributor

When you start a role, you are starting a process.

 

Let's use a concrete example.

 

* The lzo parcel contains the lzo plugin that's used by essentially all services in CDH.

* You download and activate the lzo parcel. At this point, nothing changes - you need ot restart everything to pick up the new parcel

* So you restart Hive (for example)

* When the hive server process starts, the Agent will source the lzo parcel's env script which appends the lzo jar to the classpath.

* Now the hive server is running with lzo support

 

The lzo parcel provides 'cdh-plugin' so this parcel will be loaded for all CDH processes. If a parcel provides 'hdfs-plugin' then it would only be loaded for HDFS, etc.

 

https://github.com/cloudera/cm_ext/wiki/The%20parcel.json%20file#providing-tags

avatar
Contributor
Your reply is clear to me.
When I checked the LZO example, I found " $PARCELS_ROOT" and "$PARCEL" are used as predefined environment variables, but when I echoed them, nothing come out. there is no such environment variables defined using " export -p".
I am wondering how to list all cms managed environment variables and then use them to evaluate if my script has been sourced or not?

avatar
Contributor

Where are you trying to echo variables? You won't see them set except from inside your environment script (which is sourced, so echo won't do anything). To see if your parcel's script is being sourced, look at the stderr.log for the process you think should be loading your parcel.

avatar
Contributor

I found stderr.log from CM web console. Could you tell me what's the absolute path of the log that i can track each process?

 

avatar
Contributor

On each host, you have: /var/log/cloudera-scm-agent/process/<process-name>/logs/stderr.log

avatar
Contributor

I checked this folder before, there is no "process" folder. i am using cm v5.

 

[root@lcigh158 cloudera-scm-agent]# pwd
/var/log/cloudera-scm-agent
[root@lcigh158 cloudera-scm-agent]# ls
cloudera-scm-agent.log  cmf_listener.log  supervisord.out
cloudera-scm-agent.out  supervisord.log

avatar
Contributor

Sorry, /var/run