Yarn ClassPath Value for Polybase Pushdown

[#1]

I have Polybase setup and configured to work with my HDP Sandbox. I am able to Query my ORC table successfully but when I try to use Pushdown the query fails. I setup the Pushdown by copying the yarn.application.classpath from my yarn-site.xml in the path /etc/hadoop/conf on my HDP Sandbox to the yarn-site.xml file located in the path C:\Program Files\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQL\Binn\Polybase\Hadoop\conf.

<name>yarn.application.classpath</name> <value>$HADOOP_CONF_DIR,/usr/hdp/current/hadoop-client/*,/usr/hdp/current/hadoop-client/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*</value> </property>

When I review the job log within Ambari I see the below error. Can anyone tell me what I am missing in my classpath?

Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster

[#2]

@Artem Ervits can you give me a hint as to what is wrong with my classpath?

[#3]

Go to ambari, click on download client configs, place mapred-site, core-site and hdfs-site into the same location as yarn-site.

[#4]

I am working on this suggestion.

[#5]

Artem - after adding the files into the config folder in the Polybase Path I got the below error. I checked the mapred-site.xml file and the path it is unable to parse is in the mapreduce.application.framework.path. Should I go though the file and hardcode current in place of where I see ${hdp.version}?

Java exception raised on call to JobSubmitter_SubmitJob: Error [Unable to parse '/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a URI, check the setting for mapreduce.application.framework.path] occurred while accessing external file.'

[#6]

Ok - I got past the parse error by substituting the version number and the job was submitted to my sandbox and a different error showed up this time that is below.

Application application_1465576751567_0001 failed 2 times due to AM Container for appattempt_1465576751567_0001_000002 exited with exitCode: 1 For more detailed output, check application tracking page:http://sandbox.hortonworks.com:8088/cluster/app/application_1465576751567_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_e38_1465576751567_0001_02_000001 Exit code: 1 Exception message: /hadoop/yarn/local/usercache/pdw_user/appcache/application_1465576751567_0001/container_e38_1465576751567_0001_02_000001/launch_container.sh: line 11: %PWD%;$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp/${hdp.version}/hadoop/lib/hadoop-lzo-0.6.0.${hdp.version}.jar:/etc/hadoop/conf/secure;job.jar/job.jar;job.jar/classes/;job.jar/lib/*;%PWD%/*: bad substitution

[#7]

It still says bad substitution, you check the /var/log/Hadoop, specifically yarn and mapreduce logs

[#8]

I got past the substitution error by adding the hdp version value to the map-site.xml file. I am now getting the below error. Any suggestions?

Container exited with a non-zero exit code 1 Failing this attempt. Failing the application. 2016-06-11 03:04:34,485 INFO rmapp.RMAppImpl (RMAppImpl.java:handle(767)) - application_1465576751567_0007 State change from FINAL_SAVING to FAILED 2016-06-11 03:04:34,485 WARN resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(323)) - USER=pdw_user OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1465576751567_0007 failed 2 times due to AM Container for appattempt_1465576751567_0007_000002 exited with exitCode: 1 For more detailed output, check application tracking page:http://sandbox.hortonworks.com:8088/cluster/app/application_1465576751567_0007Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_e38_1465576751567_0007_02_000001 Exit code: 1 Exception message: /bin/bash: line 0: fg: no job control Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control at org.apache.hadoop.util.Shell.runCommand(Shell.java:576) at org.apache.hadoop.util.Shell.run(Shell.java:487) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:753) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:303) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

Container exited with a non-zero exit code 1 Failing this attempt. Failing the application. APPID=application_1465576751567_0007

[#9]

Here is the Property I added.

<property> <name>hdp.version</name> <value>2.4.0.0-169</value> </property>

[#10]

Make sure everything is setup properly:

https://realizeddesign.blogspot.com/2015/09/setting-up-polybase-in-sql-server-2016.html

Also engage Microsoft support if something appears to be SQL Server related.

https://msdn.microsoft.com/en-us/library/mt163689.aspx