Member since
02-03-2015
2
Posts
1
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3450 | 02-03-2015 08:42 AM |
02-03-2015
09:15 AM
Hello, I've been trying for a couple of days to run the YARN SLS web app to profile the different schedulers. I followed the give by Apache here http://hadoop.apache.org/docs/r2.3.0/hadoop-sls/SchedulerLoadSimulator.html and was able to resolve all apparent pathing issues. However when I start the SLS app I get the exception below. I tried starting using the slsrun.sh script and using the hadoop jar option and get the same results. I have tried putting the html folder containing those files in several places defined in the hadoop classpath as well as setting the PATH variable to include the folder but I get the same result. Besides the exception the symptoms I see are the there no web pages displayed on port 10001 (which I confirmed is not in use otherwise) and in the ouput folder I get metrics files being created but no track files. Here is my command using the hadoop jar option: $ hadoop jar /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/hadoop-mapreduce/hadoop-sls-2.5.0-cdh5.3.0.jar org.apache.hadoop.yarn.sls.SLSRunner -inputrumen hadoop-sls-2.5.0-cdh5.3.0-javadoc/hadoop-sls-2.5.0-cdh5.3.0/sls/sample-data/2jobs2min-rumen-jh.json -output /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/sls_output Here is the exception I get: 15/02/03 08:58:50 INFO resourcemanager.ResourceManager: Using Scheduler: org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper java.lang.NullPointerException at org.apache.hadoop.yarn.sls.web.SLSWebApp.<init>(SLSWebApp.java:86) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.initMetrics(ResourceSchedulerWrapper.java:477) at org.apache.hadoop.yarn.sls.scheduler.ResourceSchedulerWrapper.setConf(ResourceSchedulerWrapper.java:176) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createScheduler(ResourceManager.java:272) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:422) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:914) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:234) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.sls.SLSRunner.startRM(SLSRunner.java:167) at org.apache.hadoop.yarn.sls.SLSRunner.start(SLSRunner.java:141) at org.apache.hadoop.yarn.sls.SLSRunner.main(SLSRunner.java:528) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) 15/02/03 08:58:51 INFO event.AsyncDispatcher: Registering class org.apache.hadoop.yarn.server.resourcemanager.scheduler.event.SchedulerEventType for class org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher
... View more
Labels:
02-03-2015
08:42 AM
1 Kudo
You can go here and download for a specific version : https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-sls/ The SLS scripts and config files are in the javadocs jar.
... View more