About Philip Langdale

Philip Langdale · ‎08-19-2018

Ok. Then please send me a private message with the CRN of the cluster. It's shown the cluster details.

Philip Langdale · ‎08-18-2018

Hi Cobatch, You should first retry to terminate the cluster. We sometimes see Azure that fail the termination but where it succeeds on a second attempt. If that doesn't work, we can dig deeper. Thanks.

Philip Langdale · ‎10-14-2015

Did you reboot your hosts? We've seen a problem, for which there will be a fix in the next point release, where rebooting a host causes CM to lose the parcel metadata, hence why you see this indication that the parcel information is new. As for forcing a redeployment, the simplest way is to make a config change, deploy, then change it back and deploy again. CM will skip the actual redeployment if nothing has changed.

Philip Langdale · ‎09-23-2015

Glad to hear.

Philip Langdale · ‎09-22-2015

It's there.

Philip Langdale · ‎09-22-2015

You add it in the wizard on this page at the bottom.

Philip Langdale · ‎09-17-2015

In the wizard, you can specify a custom repository to obtain the Agent from. If you choose that option, you can put in your local repo and it will be used in place of archive.cloudera.com. Of course, you must properly mirror the packages and metadata to create a compliant apt or yum repository (depending on your distro)

Philip Langdale · ‎09-17-2015

Running a local http server is the best way to handle this situation, so I'm glad you did that. We document how to set up a local parcel mirror here: http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ig_create_local_parcel_repo.html While it is technically possible to directly place files on disk and convince CM to use them, there are a lot of gotchas around it and so, this is not something we recommend any more.

Philip Langdale · ‎09-15-2015

It's worth pushing on that - I'm pretty confident that you should be able to use OpenTSDB just fine, though I'm no expert in the compatiblity constraints it has with hbase. It's definitely worth it given how old and far behind CDH4 is at this point - indendent of the specific problem you have with downloads.

Philip Langdale · ‎09-15-2015

Are you sure OpenTSDB can't work with HBase in cdh5? I see at least one discussion where it's reported to work. Where in the process is this happening? Are you using the Add Cluster wizard or trying to download it for an existing cluster? Is the configured parcel repo url in your CM settings valid?

Philip Langdale · ‎04-15-2015

@williamstw wrote: So, I suppose a more succinct question would be... how do custom CSD control scripts get access to HADOOP_PREFIX? Are they required to export it themselves in their corresponding parcel? I would, and do, recommend you simply use ../CDH/lib/... As long as your parcel depends on CDH, then you can be sure that CDH exists and that it is present in this location. Parcels are required to name their directories the same as their logical names, so this is safe and consistent behaviour.

Philip Langdale · ‎07-31-2014

https://github.com/cloudera/cm_ext/wiki/The%20parcel.json%20file#providing-tags As it discusses, if you want your parcel to affect an existing service, it should provide the 'foo-plugin' tag for that service, or cdh-plugin if you really want to affect all of them. Only one parcel can provide the required tag ('cdh' in this case), and obviously the CDH parcel provides that. What is your parcel providing that you want it to affect all services? LZO is a highly unusual example in this regard.

Philip Langdale · ‎07-30-2014

Sorry, /var/run

Philip Langdale · ‎07-30-2014

On each host, you have: /var/log/cloudera-scm-agent/process/<process-name>/logs/stderr.log

Philip Langdale · ‎07-30-2014

Where are you trying to echo variables? You won't see them set except from inside your environment script (which is sourced, so echo won't do anything). To see if your parcel's script is being sourced, look at the stderr.log for the process you think should be loading your parcel.

Philip Langdale · ‎07-30-2014

A tmpfs will default to a max size of 50% of physical RAM but this space is not allocated until its used, and tmpfs will be paged out to swap if there is memory pressure.

Philip Langdale · ‎07-30-2014

When you start a role, you are starting a process. Let's use a concrete example. * The lzo parcel contains the lzo plugin that's used by essentially all services in CDH. * You download and activate the lzo parcel. At this point, nothing changes - you need ot restart everything to pick up the new parcel * So you restart Hive (for example) * When the hive server process starts, the Agent will source the lzo parcel's env script which appends the lzo jar to the classpath. * Now the hive server is running with lzo support The lzo parcel provides 'cdh-plugin' so this parcel will be loaded for all CDH processes. If a parcel provides 'hdfs-plugin' then it would only be loaded for HDFS, etc. https://github.com/cloudera/cm_ext/wiki/The%20parcel.json%20file#providing-tags

Philip Langdale · ‎07-30-2014

Hi Greg, It's required for distros where /var/run is mounted noexec but we use it unconditonally so that the Agent is operating in a consistent environment across all distros, and certain bits of agent functionality take advantage of this (such as our logic for cleaning out /var/run/cloudera-scm-agent, which unmounts and remounts the tmpfs). What is your concern? Do you have a particular problem you are trying to solve? Thanks,

Philip Langdale · ‎07-30-2014

The environment script is sourced as part of starting processes - it's not a global thing that happens when you activate the parcel. So, assuming you start a process after activating the parcel, and assuming your parcel's 'provides' tags match what the process' service is looking for, it should get sourced.

Philip Langdale · ‎05-06-2014

As a first step, you should restart all the CM Agents. sometimes they could (in this version) get confused under certain conditions and their internal knowledge of what parcels are present diverges from what's on disk - restarting means they'll initialise themselves based on the actual on-disk state, and should bring you back into sync.

Philip Langdale · ‎05-06-2014

So, you have 146 nodes in your cluster? Are they all healthy and have CM Agents heartbeating as they should? (As indicated on the Hosts page in CM). Can you cancel the remove operation? At a high level, I'd recommend upgrading to a newer version of CM - there have been many substantial improvements in 4.7, 4.8 and now 5.0, including increasing the detail around error reporting and handling in these situations. If the option exists, I'd highly recommend upgrading to 5.0 (which you can do independent of the version of CDH running on your cluster - as long as it's not CDH 3)

Philip Langdale · ‎01-16-2014

Hi Vlad, I'm guessing you've tried again since you posted this message, but this appears to have been caused by a transient error in the Ubuntu package archive. You had a newer version of libsasl2-modules installed and it was unaware of a matching version of libsasl2-modules-gssapi-mit. I've checked the Ubuntu package list and the new version does exist. I notice you never say that you did an apt-get update - you should always do that in these situations to esnure the latest package list is downloaded.

Philip Langdale · ‎08-02-2013

There's a lot of places where something could be getting in the way, so I'd consider the following. 0) Verify that the public DNS you're trying to use is the right one for your CM server instance 1) SSH to the CM server instance and verify that the server is indeed listening on *:7180. 2) Verify there's no linux firewall running on the CM server instance 3) Verify that port 7180 is open on the security group of the CM server instance (not the same as the Hadoop cluster's group necessarily!) 4) Try and telnet or netcat to 7180 on the instance and see if you can connect. 5) Try with a browser. Windows firewall is unlikely to be an issue as outbound connections like this aren't restricted.

Online	Offline
Last Visited	‎09-13-2018 05:15 PM

Date Registered	‎07-19-2013 11:08 AM
Last Visited	‎09-13-2018 05:15 PM
Total Messages Posted	23
Total Kudos Received	2

Re: Disable remote parcel repository in CM config

Re: Why my parcel can't be sourced or run?

Re: cm_processes question

Re: Cannot delete a cluster - (TERMINATION_GENERAL...

Re: Cannot delete a cluster - (TERMINATION_GENERAL...

Re: Cloudera Manager still shows as stale client a...

Re: Disable remote parcel repository in CM config

Re: Disable remote parcel repository in CM config

Re: Disable remote parcel repository in CM config

Re: Disable remote parcel repository in CM config

Re: Disable remote parcel repository in CM config

Re: Cloudera Manager hangs when Installing Selecte...

Re: Cloudera Manager hangs when Installing Selecte...

Re: Adding hadoop jars to CLASSPATH in CSD/Parcel

Re: Why my parcel can't be sourced or run?

Re: Why my parcel can't be sourced or run?

Re: Why my parcel can't be sourced or run?

Re: Why my parcel can't be sourced or run?

Re: cm_processes question

Re: Why my parcel can't be sourced or run?

Re: cm_processes question

Re: Why my parcel can't be sourced or run?

Re: Stuck Removing Parcel

Re: Stuck Removing Parcel

Re: Installing CDH 4.4 and Manager 4.7.3 on Ubuntu...

Re: I can't connect to my public DNS (http://ec2-...