Member since
09-25-2015
46
Posts
139
Kudos Received
16
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3860 | 11-27-2017 07:37 PM | |
3046 | 09-18-2017 06:28 PM | |
1690 | 09-08-2017 06:40 PM | |
765 | 07-17-2017 07:13 PM | |
681 | 06-29-2017 06:18 PM |
04-09-2018
08:43 PM
Hi @Alexander Schätzle, We had a jira for the same issue - https://issues.apache.org/jira/browse/YARN-7269. The fix went into HDP -2.6.3.0. But I am not sure about the exact build number. May be you can try with HDP-2.6.4.0 repo.
... View more
04-09-2018
06:37 PM
Hi @Venkata Sudheer Kumar M, You can fetch total containers allocated for an application using YARN CLI - 1. Fetch applicationAttempt for the application using- yarn applicationattempt -list <applicationID>
2. Fetch all containers for that applicationAttempt using- yarn container -list <appattempt> In the step 2 above, you will also get NodeManager info on which the container was launched. From the nodemanager info on which the container was launched, you can obtain the vcores and memory allocated for that container using the below REST API curl http://<Nodemanager address>:<port>/ws/v1/node/containers/<containerID> Hope this is helpful to you!
... View more
02-02-2018
06:48 PM
1 Kudo
Hi @shashi cheppela Please find the below doc which has the info for HA in topology https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/configure_knox_for_ha.html
... View more
11-27-2017
07:37 PM
1 Kudo
Hi @Michael Bronson All services logs can be found at the location mentioned in the yarn-env.sh (can be found under Hadoop Conf files, usually /etc/hadoop/conf/) on the respective nodes. Hence please check location mentioned in yarn-env.sh on the master02 machine. Once we check the logs, it will be easier to figure out the exact reason for failure. Example yarn-env.sh export YARN_LOG_DIR=/grid/0/log/yarn/
... View more
11-02-2017
02:28 AM
1 Kudo
Hi @Antony Thilagar, The below looks like HDFS namenode is not started. Please check if HDFS components are started properly or not. Error starting ResourceManager org.apache.hadoop.service.ServiceStateException: java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) at
... View more
11-02-2017
02:28 AM
1 Kudo
Hi @Antony Thilagar, The below looks like HDFS namenode is not started. Please check if HDFS components are started properly or not. Error starting ResourceManager org.apache.hadoop.service.ServiceStateException: java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) at
... View more
09-21-2017
09:35 PM
1 Kudo
Hi @Mykyta Demeshchenko https://community.hortonworks.com/questions/75914/user-not-found-error-when-invoking-hive.html -- This may be useful for you
... View more
09-18-2017
06:28 PM
3 Kudos
Hi @PJ All services logs can be found at the location mentioned in the yarn-env.sh (can be found under Hadoop Conf files, usually /etc/hadoop/conf/) on the respective nodes.
Example yarn-env.sh export YARN_LOG_DIR=/grid/0/log/yarn/
... View more
09-18-2017
06:28 PM
8 Kudos
Hi @PJ All services logs can be found at the location mentioned in the yarn-env.sh (can be found under Hadoop Conf files, usually /etc/hadoop/conf/) on the respective nodes.
Example yarn-env.sh export YARN_LOG_DIR=/grid/0/log/yarn/
... View more
09-09-2017
10:37 PM
1 Kudo
Hi @jpj https://issues.apache.org/jira/browse/MAPREDUCE-6304 from this we can know that the feature landed in HDP 2.6 version. Please upgrade your version to HDP 2.6 More instructions on upgrade - https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_command-line-upgrade/content/ch_upgrade_2_5.html
... View more
09-08-2017
06:40 PM
11 Kudos
Hi @nur majid, In mapred queue -list output, default capacity and maximum capacity are with respect to Cluster' resources whereas current capacity is with respect to Queue's resources. Example - Assume cluster's resources - 10GB Default Queue default capacity is 20% of cluster's resources which is 2GB But default queue can go upto 30% of cluster's resources [maximum capacity] which is 3GB Current capacity is 101% of Queue's capacity [Remember , queue's capacity is 2GB] , so it will come around 2.02GB which is less than 3GB[maximum capacity].
... View more
07-17-2017
07:13 PM
5 Kudos
Hi @Anurag Mishra, If you want to change yarn-site.xml in the server, then it is also fine. But you need to restart the services on your own instead of using ambari .(if you use ambari to restart services, then as mentioned by Jay, your changes will get wiped off) WARNING : It so happens, that for some properties in yarn-site.xml, it requires restart of more than one service , in which case, you need to copy the change in all service nodes and restart them. ADVANTAGE OF AMABRI - In case of ambari, its just one point change, it will figure out which services require that change and suggest restart of all the services required.
... View more
07-10-2017
06:37 PM
1 Kudo
Hi @Sree Kupp, Can you please check ResourceManager logs and see what type error it is throwing? Please attach ResourceManager logs and capacityScheduler.xml ??
... View more
07-06-2017
06:29 PM
1 Kudo
Hi @Sebastien Chausson, There are jiras raised for the same actually. We are working on it to fix the issue.
... View more
06-29-2017
06:18 PM
7 Kudos
Hi @Rahul Gupta, Yes, a single node can have more than one application masters when we have several applications running; where each application master belongs to a unique application. None of the application master will be aware of the presence of the other. More info on AM https://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/
... View more
06-27-2017
06:44 PM
5 Kudos
Hi @Gunjan Dhawas For point 2 - Also as you mentioned its nodemanager which will communicate with container, so can nodemanager directly communicate with containers which are running on different nodes or it will go through RM to get container information. Nodemanagers are basically YARN’s per-node agent, and takes care of the individual compute nodes in a Hadoop cluster. This includes keeping up-to date with the ResourceManager (RM), overseeing containers’ life-cycle management; monitoring resource usage (memory, CPU) of individual containers, tracking node-health, log’s management and auxiliary services which may be exploited by different YARN applications.
So Nodemanagers are the nodes on which containers are launched. So yes, nodemanagers directly monitors the containers and their resource consumption. For point 1 - "The application code executing within the container then provides necessary information (progress, status etc.) to its ApplicationMaster via an application-specific protocol.", so how the application master monitor the status of containers which are running on different node than applicatioMaster. Once the applicationMaster negotiates the resources with RM, it will launch the container by providing container launch specification to the NodeManager. The launch specification includes the necessary information to allow the container to communicate with the ApplicationMaster itself. Thus ApplicationMaster gets the progress/status via application-specific protocol provided in the container launch specification
... View more
06-26-2017
07:46 PM
4 Kudos
Hi @Suhel, As mentioned in https://community.hortonworks.com/questions/35751/manage-yarn-local-log-dirs-space.html, We can safely delete files in /yarn/local if there are no applications running.
... View more
06-26-2017
06:34 PM
1 Kudo
Hi @Gunjan Dhawas, No, ApplicationMaster doesnot directly communicate with the containers. Its the nodemanagers which communicate directly with the containers. NodeManager's core part is containerManagement. ApplicationMaster, framework specific entity is responsible for negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the containers and their resource consumption More information on Nodemanager and AM - https://hortonworks.com/blog/apache-hadoop-yarn-nodemanager/ https://hortonworks.com/blog/apache-hadoop-yarn-concepts-and-applications/
... View more
06-23-2017
06:36 PM
9 Kudos
Hi @Anthony Seluk, When you disable High Availability, automatic failover also gets disabled. Hence when active RM is killed, it doesn't do automatic failover to standby RM [that is, making standby RM as active]. So there will not be any active RM; instead we have two standbyRMs. In such scenario, we need to do manual failover. More info on manual failover is given below https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html
... View more
06-21-2017
06:40 PM
1 Kudo
Hi @bhavik shah, Can you please share the capacity-scheduler.xml file? In that please check, if the default queue is stopped or not?
... View more
05-22-2017
06:33 PM
4 Kudos
Hi @priyanshu hasija, We can ignore the first two lines of exception as suggested in https://community.hortonworks.com/questions/35089/dataxceiver-error-processing-unknown-operation-jav.html. ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: infoobjects-Latitude-3550:1025:DataXceiver error processing unknown operation src: /127.0.0.1:35436 dst: /127.0.0.1:1025 java.io.EOFException: Premature EOF: no length prefix available at org.apache.hadoop.hdfs.protocolPB.PBHelper.vintPrefixed(PBHelper.java:2207) at
The following exception indicates that "kinit" is not being done. Can you please do kinit before running the hdfs copyFromLocal command ? More Info - https://community.hortonworks.com/articles/4755/common-kerberos-errors-and-solutions.html at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferServer.doSaslHandshake
... View more
04-26-2017
11:47 PM
4 Kudos
Hi @Sandeep SIngh, Each mapper and reducer is a java process and we need some reserve memory to store the java code. Hence the Map/Reducer memory should be greater than the JVM heap size.
... View more
04-14-2017
06:22 PM
6 Kudos
Hi @Carol Elliott, The applications are submitted to "Default Queue". As seen in the Yarn RM UI, "Default Queue" is full and hence it doesnt have capacity to allocate for task containers. Only Application Master container is being allocated and hence the application remains in ACCEPTED state. Please submit application to other Queue or increase capacity for Default Queue.
... View more
03-31-2017
07:09 PM
2 Kudos
Hi @Gonzalo Herreros, Did you try obtaining/renewing Kerberos ticket before trying to run webhdfs? Also since webhdfs internally uses hdfs user, can you please check the property "hadoop.kms.blacklist.DECRYPT_EEK" present in "dbks-site.xml" (/etc/ranger-kms/<version>/dbks-site.xml) under Ranger-KMS ?? If it has the value -"hdfs", that means, it will not allow hdfs to decrypt the encryption key. In that case, you need to remove "hdfs" and restart RANGER-KMS.
... View more
03-29-2017
10:25 PM
2 Kudos
Hi @dvt isoft, You cannt actually submit MR / Tez job via Yarn Webservices. Can you please try with Distributed Shell application if you want to use REST api? Or you can store your job in HDFS and use OOZIE workflow over it or use SPARK. https://community.hortonworks.com/questions/89692/can-we-submit-a-yarn-job-through-rest.html
... View more
03-29-2017
09:55 PM
5 Kudos
Hi @Aditya Mamidala, You also need to set yarn.nodemanager.localizer.cache.target-size-mb along with yarn.nodemanager.localizer.cache.cleanup.interval-ms. Also the resources are not cleaned up if they are being used by running containers. More details given in below blog https://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/
... View more
03-27-2017
06:46 PM
2 Kudos
HI @Turing nix, In Yarn Queue Manager, it says "leave blank to deny access for everyone". So i think you need not enter space as well. Attached is the screenshot for the same.screenshot.png
... View more
03-24-2017
11:52 PM
1 Kudo
Hi @dvt isoft, Can you please post the application logs and RM logs to debug further into this.
... View more
03-24-2017
09:55 PM
3 Kudos
Hi @Mahender S, Once you specify the queue as suggested Deepesh, you can also enable preemption which makes sure that the application submitted to Q2 runs even though application submitted Q1 occupies whole cluster 's resources. Capacity schedular preemption setting info can be found below: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_yarn_resource_mgt/content/preemption.html
... View more
03-23-2017
08:00 PM
1 Kudo
Can you please accept my answer if it answered your question ? 🙂
... View more