@Ryan Suarez Exit code 143 is related to Memory/GC issues. Your default Mapper/reducer memory setting may not be sufficient to run the large data set. Thus, try setting up higher AM, MAP and REDUCER memory when a large yarn job is invoked. ... View more

@Saravanan Selvam, In yarn mode you can control the total number of executors needed for an application with --num-executor option. However, if you do not explicitly specify --num-executor for spark application in yarn mode, it would typically start one executor on each Nodemanager. Spark also has a feature called Dynamic resource allocation. It gives spark application a feature to dynamically scale the set of cluster resources allocated to your application up and down based on the workload. This way you can make sure that application is not over utilizing the resources. http://spark.apache.org/docs/1.2.0/job-scheduling.html#dynamic-resource-allocation ... View more

@Bhavin Tandel, when are you hitting this errors? can you please explain the steps which lead to this error? ... View more

@Kevin Ng, can you please check the cluster configuration for Spnego authentication? Find the guidelines as below. https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.0/bk_Ambari_Security_Guide/content/ch_enable_spnego_auth_for_hadoop.html ... View more

@Juan Manuel Nieto, can you please check if a yarn/MR job related to this oozie workflow still running ? ... View more

@pooja khandelwal, what is mapred.child.java.opts property set to ? Can you please try increasing this value ? ... View more

@Kevin Ng, if you are doing kinit properly, then it can be a configuration issue related to Ranger KMS. Make sure that KMS is configured properly in your cluster. Refer to to below thread for configurations. https://community.hortonworks.com/questions/28052/exception-while-executing-insert-query-on-kerberos.html ... View more

@Kevin Ng, Follow below steps. Suppose you want to run this application as user "a". sudo su a kinit -kt <a keytab> <a principal> ./spark-submit --class org.apache.spark.examples.SparkPi --master yarn --deploy-mode client ../lib/spark-examples*.jar 100 ... View more

@n c, There can be multiple reasons for this issue. 1) Make sure that you have odd number of zookeepers. (example : 3) 2) Also make sure that the ports on which zookeepers are listening are open and used by Zks. 3) Check the firewall setting between hosts to make sure they can communicate with each other. This is also a good read. http://stackoverflow.com/questions/13316776/zookeeper-connection-error ... View more

@Bhavin Tandel, are you using HDP cluster? Can you please check if you have followed steps mentioned in below article? https://community.hortonworks.com/articles/80059/how-to-configure-zeppelin-livy-interpreter-for-sec.html ... View more