About iroberts

iroberts · ‎01-12-2017

yarn-client is not supported as per new guidelines, only yarn-cluster mode is supported within livy interpreter from Zeppelin

iroberts · ‎01-12-2017

yarn-client is not supported, only yarn-cluster is supported

iroberts · ‎01-11-2017

@elkan li which user are you logged in as when launching? Can you try an su hive then launch the hive cli? It seems the issue is with the launching user

iroberts · ‎01-11-2017

@sagar pavan the Diagnostic message indicates the Users AM resource limit is exceeded. Please review the capacity scheduler's AM resource limit and raise it from the default 20%, this should allow the AM container to be launched

iroberts · ‎12-21-2016

livy documentation states to add jars with the interpreter key value livy.spark.jars. This is applicable when running livy interpreter in yarn-cluster mode only (livy.spark.master). This can be done in local or yarn-client mode as well however by adding the jars to the /usr/hdp/<version>/livy/repl-jars directory. This will add the jars to the spark submit command run by livy when executing within a notebook in modes other than yarn-cluster.

iroberts · ‎12-13-2016

This does not look like an issue with the jar being included but rather an issue with the import statement. I breifly looked on google and see similar descriptions stating to try org.mongodb. I would focus on the import statement more than the inclusion of the jar for livy.

iroberts · ‎12-13-2016

@Mickaël GERVAIS check to make sure livy interpreter is listed in the interpreter bindings for the notebook. Also, set DEBUG on the livy server and check in the livy out file produced on the server. Finally, make sure you have restarted livy and zeppelin to pick up the changes. I tested and it did work for me.

iroberts · ‎12-13-2016

The jars should be able to be added by using the parameter key livy.spark.jars and pointing to an hdfs location in the livy interpreter settings. This does not seem to work. I had to place the needed jar in the following directory on the livy server: /usr/hdp/2.5.3.0-37/livy/repl-jars

iroberts · ‎09-29-2016

@ARUN Yes, you can use node labels and queues together. Here is some documentation regarding that: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.6/bk_yarn_resource_mgt/content/configuring_node_labels.html

iroberts · ‎09-27-2016

That is not a good solution as you are saying yarn does not need any healthy disks to function. You essentially disable the health check with this. There is an underlying problem with the disks and that is the reason for it being marked unhealthy.

Online	Offline
Last Visited	‎06-20-2019 04:30 PM

Member Since	‎09-29-2015 12:30 PM
Last Visited	‎06-20-2019 04:30 PM
Posts	58
Kudos received	26

Cloudera Community

Re: node labels and queues

Re: Yarn available memory

Re: API to manage YARN Capacity Queue

Re: Capacity Scheduler does not use all the resour...

Re: YARN doesn 't start

Re: Adding additional jars to livy interpreter wit...

Re: In Zeppelin livy interpreter, what should be v...

Re: Can't not login Hive Service with CLI?

Re: Tez job hang, waiting for AM container to be a...

Adding additional jars to livy interpreter within ...

Re: How to import External Libraries for Livy Inte...

Re: How to import External Libraries for Livy Inte...

Re: How to import External Libraries for Livy Inte...

Re: node labels and queues

Re: how to repair Unhealthy Nodemanager ??