Reply
New Contributor
Posts: 1
Registered: ‎09-10-2018

Using YARN Scheduler Load Simulator (SLS)

Based on this article, "https://hadoop.apache.org/docs/r2.7.6/hadoop-sls/SchedulerLoadSimulator.html", In a normal hadoop installation everything to config and run SLS is present in $HADOOP_ROOT/share/hadoop/tools/sls/

 

In Cloudera Installation, I didn't find the above directory for SLS, but instead found hadoop-sls.jar in CDH-5.8.3-1/lib/hadoop-mapreduce directory. I believe that using this jar with SLSRunner class is similar to running slsrun.sh with the same parameters (as mentioned in the link above).

 

But, according to the link, configuration to start SLS depends on the sls-runner.xml file present under $HADOOP_ROOT/share/hadoop/tools/sls/sample-conf. I wasn't able to find such a config xml in Cloudera's installation.

 

Could someone help me to figure out how to configure SLS in cloudera, or at least point me in the right direction.

Also, if a small example use case to run SLS in cloudera were to be given, i'd appreciate it. Thanks :)

Posts: 1,760
Kudos: 379
Solutions: 282
Registered: ‎07-31-2013

Re: Using YARN Scheduler Load Simulator (SLS)

> But, according to the link, configuration to start SLS depends on the sls-runner.xml file present under $HADOOP_ROOT/share/hadoop/tools/sls/sample-conf. I wasn't able to find such a config xml in Cloudera's installation.

The scheduler load simulator module files are not packaged as part of CDH parcels and packages. You can find them instead in the source+binary tarball at http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.8.3.tar.gz, or from the releases source repository at https://github.com/cloudera/hadoop-common/tree/cdh5.8.3-release

> I believe that using this jar with SLSRunner class is similar to running slsrun.sh with the same parameters (as mentioned in the link above).

While I've not attempted to run it with a configured input, the command you can use to invoke the SLS tool would be:

~> hadoop org.apache.hadoop.yarn.sls.SLSRunner

Overall it'd be better if you use the tarball, as then the instructions from your linked document would be closer to follow.
Announcements