I actually did figure out how to solve this, at least in our case.  Hopefully it's relevant to yours as well.  See here. ... View more

Just a note about the CM API: From what I can tell, the API doesn't bring more parameters back that what you can see in the Configuration tab for each app/service. Being able to see them in the INFO logs was exactly what was needed.  However, it would be nice to be able to use the API to get the same info as the INFO logs provides.  I could see some automation opportunities in the future... ... View more

Yes - that WIP links back to KUDU-1603 that I shared earlier. Guess, we will have to wait it out. Thanks for your response.  ... View more

>" Your cluster does sound unhappy"   LOL.  I'd say more like pi$$ed.   🙂   > If it's a JVM issue, we've seen in some cases that increasing heap sizes helps. Setting ipc.client.rpc-timeout.ms to 60000 I'd say it's more like a set of overloaded namenodes.  And, according to the research I've done so far, for another problem, I have a theory that I can use a more up to date GC and it should increase the performance and reduce the number of Zookeeper failures we have.  Our ZK failures are sporadic and happen every 2 to 3 days.  Sometimes more sometimes less.  Moving ZK to separate nodes is not an option at this point and I'm doing all I can to try to reduce the number of failures short of moving the service. I'll check our settings on this and see if we can do one or both.  I suspect we have increased JVM heap already, but not sure?   >We've also seen the file handle cache that got enabled by default in CDH5.15 help a lot in reducing namenode load, I assume this is available before this version but was not enabled by default???  I'll look it up and see...   > I agree 100%. I think whoever named it was either overly optimistic and assumed there wouldn't be a significant gap in time, or it was named from the point of view of the code rather than the external system So, my question is, is there an indicator in the  Query Details that indicates something was returned?  I know I get an initial set of results back.  Without that "fetch" metric meaning what the word actually says, I don't know what indicates how long it took to get the first set of records back???   Back to the original issue... Given that the issue appears to be the last query issued in Hue tends to show up as still executing 2.2 hours later and has already returned a count almost immediately.  Obviously, the parameters for idle timeouts for sessions and queries is not marking the session as closed.  Therefore appearing to still be executing: Is this causing resource issues because the session is being held open and appearing to be still executing?  I would assume so as it is waiting on a fetch of subsequent sets of records??? What parameter(s) will close the session from the last query executed?   Just to let you know, I've come in late to the game and am still learning CM and Cloudera Manager.  I understand a lot but with 1000s of parameters for all the apps and an ample supply of problems, it'll take a while.   🙂   Thanks for all your help.  It is nice to have a response on this forum.  The last couple posts were not very useful.  We do have a service contract and although I am one of the 3 admins, they are working on adding me to the support contract so I can put in tickets and get support directly. Until then, I appreciate the help! ... View more

I updated this response to include more information, especially with some details not mentioned in the Hue page about what happens in Impala. ... View more

Hi LiWang,   Yes I have checked the audit logs of Impala.  I ran a grep -i check looking for "OPERATION_TEXT" as a part of a SQL statement that was executed based on the error I was getting in the log and did not find any log entries with that pattern.  I used "ignore case" with grep just to make sure. Since this is a Navigator record that Impala is trying to insert I tried looking into those logs as well.  Since I don't have permission to look into those logs, I modified the URL in CM's interface so I could see the log content.  Very slow and cumbersome but never found any thing that was very helpful as far as the actual SQL statement that was failing. Another thing to note is the impala_audit_event_log_1.0-xxx files contain no INSERT statements at all.  It's hard to believe no inserts have happened using Impala at all.  The reason I was looking for an Insert command is my assumption is the error indicated it was trying to insert into the MySQL audit table and was not able to because of the character set issue. The log that showed the error did not show the whole SQL statement, unfortunately, but I did get a few characters of the offending characters in hex form and the column (OPERATION_TEXT) it we trying to insert into. Any ideas?   Thanks! ... View more

Hey pd,   Unfortunately, I'm not a Java programmer.  I do PHP, Python and bash and decided not to develop in Java many years ago as it appeared to be way too complex compared to other solutions.  Also, at that point in time, Java was still having speed issues being psuedo compiled.  The language I was using at that time was fully compilable. Also, I assumed that Cloudera would get around to it eventually, as it's a part of the tools that appear to be under the CM support umbrella.  Or, at least it's installed with the Kafka CSD. I would think some kind of CSD to generically allow scripts to be run based on service/role events would be ideal??? Thanks for finally getting back to me!   Have a great day! ... View more

Thanks! ... View more

Although I don't have this problem, I would like to know the answer to this. It would definitely be good to know. Unfortunately, I have posted questions and received nothing but crickets as it appears you have. Hopefully, someone will get back to you. ... View more

Yeah, they sold Powerschool off to a competitor who took it way past what we had.  One of the Java engineers talked the new company into letting him re-write it in Java. It looks pretty cool from the last time I looked at it.   I remember that group back in the day.  I still have my email address and am using 50GB of iCloud storage for the day my Mac goes down.  I bought a 2016 laptop in 2017 and absolutely love it.  All except the memory.  In which they fixed in the newest releases of MacBook Pro.  Dang company!   🙂   Dude, I got some stories of sticking with something.  I have about 33 years of Unix experience starting back in the day with Sun Microsystems servers.  I own a Sun Sparc Classic and a Sun Enterprise 250 workgroup server.  Both of them will still boot.   I wil let you know if anything pops up as I am going to do some Django development as a phase II project to manage users automatically.  I figured out how to web scrape the Hue page using the older user list URL (/useradmin/users/) that doesn't use Javascript to load the user list (/hue/useradmin/users/).  Although, Hue warns the older URL is old, it still works.  I have not figured out yet how to load the user list with the new (current) URL of "/hue/useradmin/users/" but will get back to it later.  It would be nice to have an API for Hue as Cloudera Manager has.  My project was to auto-delete Cloudera and Hue users that have left the company.  I have all the prototype functionality built and am about to bring it all together into a package.   Thanks for all your help! ... View more

BTW, I should have made this request in my previous post.  Please DO NOT remove the /useradmin/users/ URL option until you have come up with an API or an easier way to scrape the web pages.  Being that the message indicates that is the old way of getting the list of users and the new way is using Javascript to load it, if you remove the old way and we upgrade, using the old URL won't allow me to load the user list. So, can someone there make a note to not remove that old URL please!!! ... View more