@Javert Kirilov The config json of ats-hbase should be created on startup of ResourceManager. If it wasn't created, then check the logs of the ResourceManager for any errors related to ats-hbase while starting up. ... View more

@Sam Hjelmfelt Running Docker containers which have ENTRYPOINT in YARN Services requires additional configuration to be specified in the service spec. The env variable YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE needs to be set to true. Additionally the launch command parameters are separated with commas instead of space. Try running with the below spec. { "name": "myapp", "version": "1.0.0", "description": "myapp", "components": [ { "name": "myappcontainers", "number_of_containers": 1, "artifact": { "id": "myapp:1.0-SNAPSHOT", "type": "DOCKER" }, "launch_command": "input1,input2", "resource": { "cpus": 1, "memory": "256" }, "configuration": { "env": { "YARN_CONTAINER_RUNTIME_DOCKER_RUN_OVERRIDE_DISABLE": "true" } } } ] } For further reference, refer to the documentation here ... View more

@Prashant Gupta From your logs attached, it looks like you have enabled GPU Scheduling. But it is still using the DefaultResourceCalculator. 2018-10-22 17:48:02,490 FATAL resourcemanager.ResourceManager (ResourceManager.java:main(1495)) - Error starting ResourceManager org.apache.hadoop.yarn.exceptions.YarnRuntimeException: RM uses DefaultResourceCalculator which used only memory as resource-type but invalid resource-types specified {yarn.io/gpu=name: yarn.io/gpu, units: , type: COUNTABLE, value: 0, minimum allocation: 0, maximum allocation: 9223372036854775807, memory-mb=name: memory-mb, units: Mi, type: COUNTABLE, value: 0, minimum allocation: 1024, maximum allocation: 191488, vcores=name: vcores, units: , type: COUNTABLE, value: 0, minimum allocation: 1, maximum allocation: 32}. Use DomainantResourceCalculator instead to make effective use of these resource-types In YARN->Configs->Advanced->Scheduler , set the following yarn.scheduler.capacity.resource-calculator=org.apache.hadoop.yarn.util.resource.DominantResourceCalculator ... View more

@Eddie Generally specifying the mapreduce.map.java.opts in quotes will work for all the example jobs. The following command running a pi job worked. yarn jar /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-examples.jar pi -Dmapreduce.map.java.opts="-XX:+PrintGCDetails -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDateStamps -XX:SurvivorRatio=8" 1 1 I see that your command uses your specific class Myclass.class. The example pi job works because it parses the command line options using org.apache.hadoop.util.GenericOptionsParser Your MyClass.class should use org.apache.hadoop.util.GenericOptionsParser to parse the command line options for it to work properly. ... View more