Re: How to restart/refresh client config on all th...

shyamshaw · ‎02-28-2020

@jsensharma Thank you for your response. please check if you have enabled the LLAP interactive query in order to get the statistics for the partition table : I have configured HiveServer2 Interactive but how can I verify whether it is being enabled to get the statistics for the partition table. "Interactive Mode" was set to "false" and I changed it to "true" and did hard refresh of the browser but still the same issue.

shyamshaw · ‎02-28-2020

Hi Team, We are getting below error when clicking on Statistics tab in Hive View for partitioned table. I found one bug which seems to be relevant but this is already resolved in 2.5.0. Ambari version which I am using is 2.6.2.2. Hive view 2.0. https://issues.apache.org/jira/browse/AMBARI-20113 Any help on this issue. Regards, Shyam

shyamshaw · ‎01-06-2020

@Shelton Sorry for late response. I followed the thread and was able to configure Fair Scheduler successfully. Thank you for your help. Cheers.

shyamshaw · ‎01-02-2020

Hi @Shelton Thank you for the update. It seems there are some configuration needs to be added/changed in YARN. I will perform the changes and update you in case of any failures.

shyamshaw · ‎01-02-2020

Hi Team, I am trying to use FairScheduler in HDP 3.0.1. As per https://stackoverflow.com/questions/38208845/switching-from-capacity-scheduler-to-fair-scheduler-on-hortonworks-data-platform it seems to be possible in HDP 3.0. I have changed the schedule claa and set yarn.scheduler.fair.allocation.file in custom yarn-site in ambari (screenshot attached). But when I restart ResourceManager it is failing with below error. 2020-01-02 16:21:54,134 INFO fair.AllocationFileLoaderService (AllocationFileLoaderService.java:reloadAllocations(230)) - Loading allocation file file:/etc/hadoop/conf/fair-scheduler.xml 2020-01-02 16:21:54,221 INFO monitor.SchedulingMonitor (SchedulingMonitor.java:serviceInit(60)) - Initializing SchedulingMonitor=SchedulingMonitor (ProportionalCapacityPreemptionPolicy) 2020-01-02 16:21:54,221 INFO capacity.ProportionalCapacityPreemptionPolicy (ProportionalCapacityPreemptionPolicy.java:init(160)) - Preemption monitor:org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy 2020-01-02 16:21:54,222 INFO service.AbstractService (AbstractService.java:noteFailure(267)) - Service SchedulingMonitor (ProportionalCapacityPreemptionPolicy) failed in state INITED org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Class org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler not instance of org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.init(ProportionalCapacityPreemptionPolicy.java:165) at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.serviceInit(SchedulingMonitor.java:61) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitorManager.updateSchedulingMonitors(SchedulingMonitorManager.java:93) at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitorManager.initialize(SchedulingMonitorManager.java:123) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.serviceInit(FairScheduler.java:1422) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:817) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1230) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:317) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1512) Any help on this issue.

shyamshaw · ‎12-10-2019

Couldn't find a better way than running an ansible command to find out stale llap daemons and kill the process. ansible -m shell -a 'ps -ef | grep -i llap' -i hosts.ini workr_nodes Note : No restart required for HiveServer2 Interactive.

shyamshaw · ‎12-09-2019

HiveServer2 Interactive UI is showing Launching Instances. This seems to be stale llap deamons. How can we find these deamon process running on which server as we have around 300 nodes?

shyamshaw · ‎12-05-2019

@prabhat_ratnala you can also try to disable cbo and then run the query. This worked for me. set hive.cbo.enable=false;

shyamshaw · ‎11-18-2019

@sagarshimpi Thank you.

shyamshaw · ‎11-18-2019

@sagarshimpi Thank you for the link. Is it okay to remove the PIG clients if there no such use cases. Will this impact any of the service in the cluster.

shyamshaw · ‎11-18-2019

Hi Team, Could you please help me to understand why installing Hive Service requires PIG clients to be installed. Also, is it okay to remove PIG clients after installing hive.

shyamshaw · ‎10-16-2019

@ngarg - Thank you for the confirmation.

shyamshaw · ‎10-15-2019

Hive's execution engine supports MapReduce & Tez. Does it supports Spark in HDP 3.x. https://issues.apache.org/jira/browse/HIVE-7292

shyamshaw · ‎09-16-2019

I was able to figure out with the help of below community article. https://community.cloudera.com/t5/Community-Articles/How-to-install-all-clients-on-all-nodes-in-your- cluster/ta-p/246821 script.sh --------- curl -s -u admin:admin http://ambari:8080/api/v1/hosts|grep host_name|sed -n 's/.*"host_name" : "$[^\"]*$".*/\1/p'>hostcluster.txt while read line ; do j=$line mkdir -p $j curl -s -u admin:admin -H "X-Requested-By:ambari" -i -X PUT -d '{"RequestInfo":{"context":"Restart HDFS Client","operation_level":{"level":"HOST_COMPONENT","cluster_name":"rest","host_name":"$j","service_name":"HDFS"}},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://ambari:8080/api/v1/clusters/rest/hosts/$j/host_components/HDFS_CLIENT done < hostcluster.txt

shyamshaw · ‎09-16-2019

Below article shows how to restart clients on Specific host. https://cwiki.apache.org/confluence/display/AMBARI/Restarting+host+components+via+the+API Is there any way we can perform the same on all the hosts in the cluster at one time. Thank you in advance.

shyamshaw · ‎09-11-2019

Hi @jsensharma Thank you for your response. I checked the Grafana Admin and the version is v2.6.0. In Dependencies, Prometheus 1.0.0 is also mentioned as DataSource. Do we need to configure other DataSource in newer version of Grafana for this feature or can we use AmbariMetrics as DataSource.

shyamshaw · ‎09-10-2019

Hi Team, I am trying to create a custom metric and import NFS metrics into Grafana but it doesn't seems to be working. Followed, https://hadoop.apache.org/docs/r2.8.0/hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html and now metrics is available at http://<hostname>:50079/jmx as a JSON document. <property> <name>nfs.metrics.percentiles.intervals</name> <value>100</value> </property> Also, tried to import json as mentioned in below link but no luck. https://grafana.com/grafana/dashboards/6257 HDP - 2.6.5 Ambari - 2.6.2.2 Any help on this.

shyamshaw · ‎05-22-2019

This seems to be related to https://issues.apache.org/jira/browse/YARN-3432

shyamshaw · ‎05-17-2019

Hi All, We have observed that whenever VCores Total equals to VCores Used, total memory in RM UI shows incorrect memory. We have checked and all the NodeManagers are up. Is this a expected behavior/Bug? Regards, Shyam

shyamshaw · ‎01-07-2019

@siva s please refer - https://community.hortonworks.com/questions/91199/yarn-web-interface-reporting-0-for-many-metrics-fo.html

shyamshaw · ‎05-09-2018

Hi All, We are trying to find out a way from where we can get the information of applications ran on YARN over a period of time. Let's say in a week. We tried with YARN/APP Timeline Server rest api's but that didn't helped much. We have AMS configured. In YARN - HOME, under Apps we can view the completed application on a day to day basis. We tried accessing hbase using phoenix client to capture the same but with no luck. Any help on this.

shyamshaw · ‎03-22-2018

Hi @apatel can you validate the custom yarn-site properties once again. yarn.timeline-service.http-authentication.proxyuser.<AMBARI_SERVER_PRINCIPAL_USER>.groups=* yarn.timeline-service.http-authentication.proxyuser.<AMBARI_SERVER_PRINCIPAL_USER>.hosts=* yarn.timeline-service.http-authentication.proxyuser.<AMBARI_SERVER_PRINCIPAL_USER>.users=*

shyamshaw · ‎02-15-2018

@Sanjeev Rao check this - https://issues.apache.org/jira/browse/KNOX-728. This could be related.

shyamshaw · ‎01-16-2018

Hi, @Pranav Lal We have changed the owner for hive.service.keytab (under /etc/security/keytabs) from root:root to hive:hadoop. Generated ticket using kinit and tried the beeline command which worked fine from terminal. As discussed, the issue still persists when the user is trying to connect to hive using beeline connection string using %%bash function (in JupyterHub notebook).

shyamshaw · ‎01-10-2018

@Aditya Sirna Thanks for the information.

shyamshaw · ‎01-10-2018

@Aditya Sirna Thanks for the response. Could there be any downside of skipping the service check during rolling upgrade.

shyamshaw · ‎01-10-2018

We are now sure that we can't remove the clients from Datanodes as our applications will start failing. So, is there any way we can skip the service checks during the rolling upgrade process.

shyamshaw · ‎01-09-2018

Hello All, We have a cluster with 160+ nodes. We are planning for an rolling upgrade from HDP 2.5 to 2.6 using ambari. We have edgenodes where all the clients has been installed. In addition, we also have clients installed on all the datanodes (150+ nodes) like HDFS client, Zookeeper client etc. If you review the below link, we could see that ambari does validate the services multiple times by running service checks during upgrade process. https://docs.hortonworks.com/HDPDocuments/Ambari-2.6.0.0/bk_ambari-upgrade/content/upgrading_HDP_perform_rolling_upgrade.html This will consume a lot of time during the upgrade. We can minimize the time by limiting the clients service checks. Do we really need all the clients to be installed on the datanodes? If we remove the clients from datanodes, will there be any impact on the cluster in terms of functionality. If there is no impact then what all clients can be removed . Appreciate your input!!

shyamshaw · ‎11-10-2017

Hi, @Jay Kumar SenSharma Surprisingly, now I am able to connect to Hiveserver2 using Database Type as Apache Hive2 also. I believe it is picking up the correct driver from /opt/yellowfin/appserver/webapps/ROOT/WEB-INF/lib directory.

shyamshaw · ‎11-10-2017

Hi, @Jay Kumar SenSharma I was able to fix the issue. Now we are able to connect to Hiveserver2. -> Used hive-jdbc-1.2.1000.2.6.2.0-205-standalone.jar instead hive-jdbc-1.2.1000.2.6.2.0-205.jar -> Selected Database, Generic JDBC instead Apache Hive2 as Apache Hive2 was using com.hof.mi.jdbc.driver.HiveJDBC as default driver and it was not allowing me to change driver to org.apache.hive.jdbc.HiveDriver -> Connection String : jdbc:hive2://<Hiveserver2 hostname>:10000/default -> Username/Password : blank -> Test Connection (It worked this time), Create View & Analyze.

Online	Offline
Last Visited	‎02-28-2020 06:33 AM

Date Registered	‎10-26-2016 06:42 AM
Last Visited	‎02-28-2020 06:33 AM
Total Messages Posted	98
Total Kudos Received	11

Re: How to restart/refresh client config on all th...

Re: Receiving Error "Incorrect string value: '\xC5...

Re: Keys Backup other than PostgreS DB backup

Re: We are getting below error when trying to read...

Re: Hive View 2.0 : Statistics not supported for p...

Hive View 2.0 : Statistics not supported for parti...

Re: Switching from Capacity Scheduler to Fair Sche...

Re: Switching from Capacity Scheduler to Fair Sche...

Switching from Capacity Scheduler to Fair Schedule...

Re: HiveServer2 Interactive UI is showing Launchin...

HiveServer2 Interactive UI is showing Launching In...

Re: LLAP vertex errors

Re: Why Installing Hive Service requires PIG clien...

Re: Why Installing Hive Service requires PIG clien...

Why Installing Hive Service requires PIG clients t...

Re: Does Hive on Spark supported in HDP 3.x

Does Hive on Spark supported in HDP 3.x

Re: How to restart/refresh client config on all th...

How to restart/refresh client config on all the ho...

Re: How to import NFS metrics into Grafana

How to import NFS metrics into Grafana

Re: YARN UI is showing incorrect memory

YARN UI is showing incorrect memory

Re: why https://domain/ws/v1/cluster/apps displayi...

Is there a way we can find out total number of app...

Re: Ambari hive view fails with Usernames not matc...

Re: Not able to view yarn and JH logs in Knox

Re: Beeline not working through Jupyterhub termina...

Re: Question regarding client installation/deletio...

Re: Question regarding client installation/deletio...

Re: Question regarding client installation/deletio...

Question regarding client installation/deletions o...

Re: Yellowfin connection to Hiveserver2 is failing...

Re: Yellowfin connection to Hiveserver2 is failing...