@Ankit Tripathi The best way is to enable debug in the file /usr/hdp/current/ranger-admin/ews/webapp/WEB-INF/log4j.xml Just replace all lines with - <priority value="info" /> ----> <priority value="debug" /> Restart ranger. Try test connection and same time do "tail -f /var/log/ranger/admin/xa-portal.log" ... View more

@atul kumar If you run the job it will give you job_id. If the job_id is something like job_<12312324233242> then assume that oyur job is running in HDFS mode. If the job_id is job_local** then its local mode. ... View more

@Ankit Tripathi Can you check below properties if any change - <property> <name>hadoop.kms.authentication.type</name> <value>kerberos</value> </property> <property> <name>hadoop.kms.authentication.kerberos.keytab</name> <value>${user.home}/kms.keytab</value> </property> <property> <name>hadoop.kms.authentication.kerberos.principal</name> <value>HTTP/localhost</value> </property> <property> <name>hadoop.kms.authentication.kerberos.name.rules</name> <value>DEFAULT</value> </property> <property> <name>hadoop.kms.proxyuser.#USER#.users</name> <value>*</value> </property> <property> <name>hadoop.kms.proxyuser.#USER#.groups</name> <value>*</value> </property> <property> <name>hadoop.kms.proxyuser.#USER#.hosts</name> <value>*</value> </property> Make sure you have policy for user to get keys in ranger kms admin UI. ... View more

@Tajinderpal Singh A job stuck in accepted state on YARN is usually because of free resources are not enough. You can check it at http://resourcemanager:port/cluster/scheduler : if Memory Used + Memory Reserved >= Memory Total , memory is not enough if VCores Used + VCores Reserved >= VCores Total , VCores is not enough It may also be limited by parameters such as maxAMShare Follow the blog -http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/ This describes in detail how to set the parameters for YARN containers. Check below parameters - 1) yarn-site.xml yarn.resourcemanager.hostname = hostname_of_the_master yarn.nodemanager.resource.memory-mb = 4000 yarn.nodemanager.resource.cpu-vcores = 2 yarn.scheduler.minimum-allocation-mb = 4000 2) mapred-site.xml yarn.app.mapreduce.am.resource.mb = 4000 yarn.app.mapreduce.am.command-opts = -Xmx3768m mapreduce.map.cpu.vcores = 2 mapreduce.reduce.cpu.vcores = 2 ... View more

@prathap t You cannot apply patch. The BUG will be taken care in upcomming version of HDP. There might be relavent BUG filed in hortonworks jira. ... View more

@Venkadesh Sivalingam Can you check in logs if you see any error [Check in both Yarn and NM logs] usually search for parameters -org.apache.hadoop.yarn.logaggregation I see there are few bugs already with log aggregation which are fixed in HDP 2.2 and ahead- BUG-12006 https://issues.apache.org/jira/browse/YARN-2468 What is the version of HDP you are using ? Also Can you make sure those property are in place and set correctly - PROPERTIES RESPECTED WHEN LOG-AGGREGATION IS ENABLED yarn.nodemanager.remote-app-log-dir: This is on the default file-system, usually HDFS and indictes where the NMs should aggregate logs to. This should not be local file-system, otherwise serving daemons like history-server will not able to serve the aggregated logs. Default is /tmp/logs. yarn.nodemanager.remote-app-log-dir-suffix: The remote log dir will be created at {yarn.nodemanager.remote-app-log-dir}/${user}/{thisParam}. Default value is “logs””. yarn.log-aggregation.retain-seconds: How long to wait before deleting aggregated-logs, -1 or a negative number disables the deletion of aggregated-logs. One needs to be careful and not set this to a too small a value so as to not burden the distributed file-system. yarn.log-aggregation.retain-check-interval-seconds: Determines how long to wait between aggregated-log retention-checks. If it is set to 0 or a negative value, then the value is computed as one-tenth of the aggregated-log retention-time. As with the previous configuration property, one needs to be careful and not set this to low values. Defaults to -1. yarn.log.server.url: Once an application is done, NMs redirect web UI users to this URL where aggregated-logs are served. Today it points to the MapReduce specific JobHistory. ... View more