About sbpothineni

sbpothineni · ‎07-17-2019

Any "manually" killed application not showing up in the history server. In resource manager I am not able to browse the tasks. We care using Cloudera 5.14.X Any application killed by yarn does show up in history server and able to browse tasks in resource manager.

sbpothineni · ‎05-13-2019

Thanks for the reply. I already tried yarn cli, its giving only the AM container log, seems it couldn't aggregate the other containers. when a job is killed from Hue, job State: KILLED, FinalStatus: KILLED, should it complete the log aggregation for killed application?

sbpothineni · ‎05-03-2019

I kiiled Yarn mapreduce application manually, after 50% mappers finished, I can see the mappers in Resource Manager UI while the job is in porgress, but after I killed application, I can not see the tasks window any more "Tracking URL: History" not progressing to next window. Same time this application not showing up in history server. Using Cloudera enterprise 5.14.0 How can I diagnose issue, I ddin't see any exception in logs? Also how can get rid of following message: 2019-05-03 04:58:32,835 INFO org.apache.hadoop.yarn.server.webproxy.WebAppProxyServlet: dr.who is accessing unchecked http://XXXm:44182/ws/v1/mapreduce/jobs/job_1551283647797_40054 which is the app master GUI of application_1551283647797_40054 owned by spothineni

sbpothineni · ‎11-01-2018

@Harsh J A couple of more questions: http://<rm http address:port>/ws/v1/cluster/apps?queue=root.queue1 filter by queue in resource manager rest interface only showing 'running' applications only, is there any way to show finsihed applications? eve mentioning 'states=running,finished' it still showing only running applications. We are using Cloudera 5.14. Is it possible to upgrade from CDH 5.14 to CDH 6.0 with out complete reinstall?, we are using parcels.

sbpothineni · ‎10-30-2018

Sorry for reposting.. We have a strange request from users, There should be authentication for resource manager web UI, and user should able to see only his jobs. I know this is same as Hue job browser, how can I achieve with Resource amanger UI? Cluster is not kerborised. like: http://localhost:8088/cluster?user.name=testuser should be asking password, simple authentication, right now it just replaces Mr.who with 'testuser', also need to filter the jobs by user

sbpothineni · ‎10-09-2018

What is Simple Athentication here: http://rm-host:8088/cluster?user.name=asif not asking password. and RM UI should show only jobs belong to Asif. but both are not happening, How can I achieve these, can I check LDAP credentials with Simple Authentication?

sbpothineni · ‎09-11-2018

Thanks for the help, proper implmentation of readfields solved the problem.

sbpothineni · ‎09-10-2018

@Harsh J you are right readFields is just a dummy, but getLocations() is fine, we are using fair scheduler, which should take getLocations() and schedule the job to according to data localituy right?

sbpothineni · ‎09-07-2018

>>> your custom split class' readFields method I will investigate this, update this thread, thanks

sbpothineni · ‎09-07-2018

I am running a mapreduce job, calculating the split locations and I see job.split and job.splitmetainfo files contain the locations, but mapper side it is prining the locations null CDH enterprise 5.14.0 Sep 7, 9:16:49.300 AM INFO org.apache.hadoop.mapred.MapTask Processing split: AccInputSplit [splitId=174, locations=[null, null, null, null, null, null, null, null, null, null], splitLength=516537674] anybody seen like this?

Online	Offline
Last Visited	‎11-23-2020 09:03 AM

Member Since	‎07-20-2016 10:32 AM
Last Visited	‎11-23-2020 09:03 AM
Posts	40
Kudos received	3

Cloudera Community

Re: kileld application not showing up in history s...

Re: kileld application not showing up in history s...

kileld application not showing up in history serve...

Re: Resource Manager UI authentication and filter ...

Resource Manager UI authentication and filter by u...

Re: Username and Password for Resource Manager Web...

Re: MR job Split locations null

Re: MR job Split locations null

Re: MR job Split locations null

MR job Split locations null