Member since
01-15-2016
28
Posts
6
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4269 | 03-23-2017 07:37 PM | |
1676 | 07-28-2016 10:07 PM |
03-23-2017
07:37 PM
3 Kudos
A colleague once ran into something similar on HDP 2.3.x (seen in this older version only), and did the following: === After we added the following properties to Ranger KMS (via Ambari, Ranger KMS Config tab) and restarted cluster services, it appears that HAWQ is running successfully on YARN. hadoop.kms.proxyuser.rm.users=* hadoop.kms.proxyuser.rm.hosts=* This allowed us to get past the AUTHORIZATION error in Ranger KMS logs due to renewer rm user not being allowed as a proxy to get delegation token for accounts.
... View more
01-27-2017
07:11 PM
How to use Ambari View for HAWQ Query Monitoring?
This beta version of Ambari HAWQ View provides live query monitoring directly in Ambari UI, displaying a list of queries running in a HAWQ cluster. This allows admins to see the database activity on the UI in a user-friendly interface. The view front-end is based on EmberJS framework. The Java backend extends the framework provided by the ambari-views project. The front-end polls the REST API every 5 seconds, querying the pg_stat_activity view and rendering reformatted data on the UI. How to Deploy:
Deployment simply requires copying the hawq-view jar to the ambari-server host and restarting ambari-server.
ssh into ambari_server_host
$ cd /var/lib/ambari-server/resources/views/
$ curl -O https://s3-us-west-2.amazonaws.com/pivotal-ambari/stable/hawq-view/hawq-view-1.0.0.0-1.jar
$ ambari-server restart
Creating an Instance of HAWQ View:
a) The HAWQ View instance connects to the HAWQ Master through JDBC. Ensure that HAWQ pg_hba.conf is updated to allow JDBC connections from the ambari_server_host for the HAWQ View backend user account with access to pg_stat_activity view (you may use use gpadmin out-of-the-box for demo purposes).
SSH into the HAWQ Master host and update pg_hba.conf (in HAWQ Master Directory, e.g /data/hawq/master) to allow connections. Example of entry needed in pg_hba.conf, where 192.168.64.101 is the ambari_server_host and gpadmin is used to run the HAWQ View instance:
host all gpadmin 192.168.64.101/32 trust
Restart HAWQ Master via Ambari or CLI for pg_hba.conf changes to take effect.
b) Ensure that the steps above have been completed before proceeding. In Ambari UI, navigate to Admin > Manage Ambari > Views page. You should see a HAWQ view in the list. Click on Create Instance under HAWQ View. Under Settings section, provide the HAWQ username and password of the backend user account with access to pg_stat_activity view (the same user that was added to pg_hba.conf above).
Once saved, the view will be created and operational. Submit some (long running) queries and ensure they are displayed on the HAWQ View. You may hover over the queries to see the SQL statements. HAWQ Sandbox
You can always download a Sandbox VM with HAWQ pre-installed here: https://network.pivotal.io/products/pivotal-hdb Feedback
Please give this Ambari View a try and provide feedback by posting your comments below.
... View more
Labels:
11-16-2016
06:06 AM
There could be a permission issue with the user account running Ambari installation and not having permission to delete directory /var/run/ambari-server/stack-recommendations/1. Check the directory permissions and try a user with permissions on the Ambari directory. I haven't seen this error message before.
... View more
09-02-2016
06:25 PM
2 Kudos
I'll just add two more bits that I have seen some people run into often. Hopefully this is helpful in similar scenarios. 1) /etc/ssh/sshd_config file: PasswordAuthentication should be set to yes 2) In AWS, /etc/cloud/cloud.cfg should have ssh_pwauth set to True (or 1, I think). Otherwise, cloud init keeps disabling PasswordAuthentication options in ssh config.
... View more
08-25-2016
07:10 PM
@Ancil McBarnett I'm curious as to why you ran into this issue. Ambari should have automatically created gpadmin account and exchanged keys for you. Unless you manually created gpadmin account, and gave a different password on HAWQ config screen in Ambari during install? http://hdb.docs.pivotal.io/hdb/install/install-ambari.html
... View more
08-01-2016
05:17 PM
@Eric Lemmon What tutorial link are you using? If you are using this: http://hortonworks.com/hadoop-tutorial/getting-started-with-pivotal-hawq-on-hortonworks-sandbox/ ... then note that the above mentioned tutorial was designed for HAWQ 1.3, and is not a good one to use with 2.0 since a lot has changed. While a new tutorial is being crafted, you can try a few things previously posted by @Ali Bajwa and @Zack Odom: https://community.hortonworks.com/repos/48038/lending-club-demo.html https://community.hortonworks.com/articles/34193/install-hdb-hawq-via-ambari-and-use-zeppelin-for-v.html Finally, yes, the link above you mentioned a good place to start for using basic HAWQ commands, many of the commands are available via Ambari as well.
... View more
07-28-2016
10:07 PM
The links have been fixed. Now there is one generic home page link. Thanks!
... View more
07-14-2016
04:36 AM
Are you talking about an error with the installation or an issue with the documentation site? Docs are here: http://hdb.docs.pivotal.io Btw, HDP 2.5 is not supported yet.
... View more