i am using CM and CDH 5.3.3. In the CM UI for the Yarn service the configruation shows up as stale even after I deploy the configuration.
I deploy and restart the cluster, but the status is still showing up as stale :-(.
It looks like the client configurations are getting installed on the nodes (/etc/* location).
So my main questions is: How is the "stale indicator" determined?
How does it do the "diff"?
Is it something in the CM DB that needs to be reset?
Some other background:
The /tmp folder and root folder (/etc) on a data node ran out of space. I changed the config file and tried to redeploy the config before removing the /tmp data. So is there something on that data node that did not get cleaned up correctly?
Also, we put some of the gateway roles on the data nodes. That allows the Oozie shell jobs to access the client configruations. Is the order of the deployments of the client configurations causing them to show up as stale?
No answer to this ?
I'm seeing similar behaviour on a new CDH 5.4.4 cluster with the SOLR configuration. I don't seem to be able to clear the "stale" indication in the CM admin interface. I haven't seen this before with previouos versions of CDH.
I have seen this on 2 of our clusters. On one, I removed the "gateway" role for Hive on our edge node. Then I added it back and it cleared up the stale indicator (this was a POC environment).
The other environment is a production cluster. I guess I need to schedule some down time and remove/add the gateways to the nodes to see if that clears things up.
I was hoping someone could tell us how CM determines the files are stale and that would allow me to see what is going on or what I did wrong :-).
Yep, been looking at logs and such but there's nothing obvious anywhere. After your suggestion, I just removed the SOLR gateway, re-deployed config and the status is now updated.
In my case I also noticed the latest CM is 5.4.3 whereas CDH itself was updated to the latest (via parcels) which is 5.4.4. Not sure whether it matters but usually CM version >= CDH version for things to work properly.
+1 I had the same issue with CM 5.4.3 and CDH 5.4.4
I just upgraded everything to 5.4.5, but the issue is still there.
I also tried setting up a brand new 5.4.5 cluster (directly installing 5.4.5 from parcel repo, not upgrading existing install), but the issue is still there.
This is fairly annoying, but I cant seem to find anything at all in the logs to indicate why this is happening. The stale config warning is
always indicating that any number of services are missing the following line.
I did find that stopping all cluster services, doing a hard reboot of all of the systems, and redeploying the configs
before starting the services back up did hide this notification on Hive and HBase, but no luck on solr.
I also had similar stale configuration issue on CM 5.4.6 and CDH 5.4.5.Looking for solution.
Did you reboot your hosts? We've seen a problem, for which there will be a fix in the next point release, where rebooting a host causes CM to lose the parcel metadata, hence why you see this indication that the parcel information is new.
As for forcing a redeployment, the simplest way is to make a config change, deploy, then change it back and deploy again. CM will skip the actual redeployment if nothing has changed.