Member since
09-17-2015
436
Posts
736
Kudos Received
81
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 5202 | 01-14-2017 01:52 AM | |
| 7538 | 12-07-2016 06:41 PM | |
| 8965 | 11-02-2016 06:56 PM | |
| 2872 | 10-19-2016 08:10 PM | |
| 7357 | 10-19-2016 08:05 AM |
01-23-2016
01:02 AM
3 Kudos
+ @Jean-Philippe Player Partner team have built some security workshops that show authentication, authorization, audit, encryption on HDP that might be helpful:
For IPA, see here for prebuilt VM and steps on single node. @David Streever updated here for multi-node For OpenLDAP/KDC, we have similar steps here but they are not really integrated. I took another shot at this to better integrate the two and came up with the steps here but still needed to manually create principal in keytabs. Would be great to get this updated to a more complete solution (any volunteers?) For demo purposes we also have Ambari services for KDC, OpenLDAP which can be installed either on existing cluster or brought up on new cluster (via blueprints). Steps for those provided here Also note that in Ambari 2.2.0.0 onwards there is a feature to enable kerberos via blueprints (tech preview feature)
... View more
01-22-2016
11:59 PM
1 Kudo
It might be due to missing home directories in HDFS. Depending on how you run the queries, you can try running the below: #needed to run queries from Hive view in Ambari
sudo -u hdfs hdfs dfs -mkdir /user/admin
sudo -u hdfs hdfs dfs -chown admin /user/admin
#needed to run hive from CLI, when logged in as root
sudo -u hdfs hdfs dfs -mkdir /user/root
sudo -u hdfs hdfs dfs -chown root /user/root
#needed to run beeline from CLI
sudo -u hdfs hdfs dfs -mkdir /user/anonymous
sudo -u hdfs hdfs dfs -chown anonymous /user/anonymous
... View more
01-22-2016
07:31 PM
3 Kudos
@Hammad Ali I tried this recently on kerborized cluster and ran a test topology as below: storm jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-topologies-*.jar storm.starter.WordCountTopology WordCountTopology -c localhost I got the below authorization error: Caused by: AuthorizationException(msg:getClusterInfo is not authorized) Then after creating valid Ranger policy for Storm it worked fine You can find my steps and screenshots here: https://github.com/abajwa-hw/security-workshops/blob/master/Setup-ranger-23.md#setup-storm-plugin-for-ranger Could you check the above steps and double check in your env: that kerberos is enabled the Storm plugin for Ranger was successfully installed (from Ranger UI) and check the steps above? If it still does not work, you may need to check what the nimbus.authorizer is set to using the command below (if its set to "SimpleACLAuthorizer" there may be something wrong with the setup) cat /etc/storm/conf/storm.yaml | grep nimbus.authorizer
... View more
01-22-2016
04:54 PM
@mkataria perhaps you would consider posting this as an article instead of a question? It would be a useful article because others have run into this in the past (see here)
... View more
01-22-2016
08:24 AM
2 Kudos
HDP 2.3.4 is currently the latest GA release of HDP and comes with Hive 1.2.1.2.3. You can install this using Ambari 2.2.0 Official steps available in Ambari docs here Automation to easily install Ambari on test/dev environments available here
... View more
01-22-2016
08:21 AM
1 Kudo
Currently the latest sandbox is based on HDP 2.3.2 and is available from http://hortonworks.com/sandbox @rmolina confirmed that the next sandbox that will be available will be the release following HDP 2.3.4 (likely HDP 2.4). If you like you can build you own HDP 2.3.4 sandbox my installing Ambari 2.2.0 on a local VM (a way to do this easily in only a few commands is to use Ambari bootstrap)
... View more
01-22-2016
02:02 AM
1 Kudo
@Siva Konda Make sure you are accessing Ranger Admin UI on port 6080 e,g, http://<VMs IP>:6080 If you are using VirtualBox, make sure that port 6080 has been forwarded (see here on how) Make sure Ranger is listening on port 6080 netstat -tulpn | grep 6080 If not, try restarting Ranger service sudo service ranger-admin restart
Finally check the Ranger logs (e.g. xa_portal.log and catalina.out) under below dir: ls -la /var/log/ranger/admin/
... View more
01-20-2016
06:58 AM
2 Kudos
Check the "Single view" demo/workshop repo we put together here that show examples of both bulk import and incremental update from Mysql to Hive using staging/final tables: https://community.hortonworks.com/content/repo/10062/single-view-demo.html
The one you are looking for is the Mysql version, available here: https://github.com/abajwa-hw/single-view-demo/blob/master/singleview-mysql-advanced-23.md
... View more
01-19-2016
02:10 AM
Thanks @Scott Shaw...updated
... View more
01-19-2016
02:10 AM
Thanks @Artem Ervits! Updated it
... View more