Hi @Gunjan Dhawas, No, ApplicationMaster doesnot directly communicate with the containers. Its the nodemanagers which communicate directly with the containers. NodeManager's core part is containerManagement. ApplicationMaster, framework specific entity is responsible for negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the containers and their resource consumption More information on Nodemanager and AM - https://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/ https://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/ ... View more

Hi @priyanshu hasija, We can ignore the first two lines of exception as suggested in https://community.hortonworks.com/questions/35089/dataxceiver-error-processing-unknown-operation-jav.html. ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: infoobjects-Latitude-3550:1025:DataXceiver error processing unknown operation src: /127.0.0.1:35436 dst: /127.0.0.1:1025 java.io.EOFException: Premature EOF: no length prefix available at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2207) at The following exception indicates that "kinit" is not being done. Can you please do kinit before running the hdfs copyFromLocal command ? More Info - https://community.hortonworks.com/articles/4755/common-kerberos-errors-and-solutions.html at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake ... View more

Hi @Sandeep SIngh, Each mapper and reducer is a java process and we need some reserve memory to store the java code. Hence the Map/Reducer memory should be greater than the JVM heap size. ... View more

Hi @dvt isoft, You cannt actually submit MR / Tez job via Yarn Webservices. Can you please try with Distributed Shell application if you want to use REST api? Or you can store your job in HDFS and use OOZIE workflow over it or use SPARK. https://community.hortonworks.com/questions/89692/can-we-submit-a-yarn-job-through-rest.html ... View more

Hi @Aditya Mamidala, You also need to set yarn.nodemanager.localizer.cache.target-size-mb along with yarn.nodemanager.localizer.cache.cleanup.interval-ms. Also the resources are not cleaned up if they are being used by running containers. More details given in below blog https://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/ ... View more

Hi @dvt isoft, Can you please post the application logs and RM logs to debug further into this. ... View more

Can you please accept my answer if it answered your question ? 🙂 ... View more