Member since
12-21-2015
43
Posts
10
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3451 | 12-08-2016 12:33 AM | |
6666 | 01-29-2016 08:44 PM | |
4444 | 01-28-2016 10:48 PM |
04-07-2017
07:52 PM
I added: export ZEPPELIN_MEM="-Xms1024m -Xmx2048m -XX:MaxMetaspaceSize=512m" The out of memory issue was fixed; however, I got another new issue related to yarn configuration. I will ask it with "new question". Thanks!
... View more
02-11-2017
05:28 AM
1 Kudo
@Shigeru Takehara, You can definitely have different nodes with different memory/cpu in a cluster. You can have a 5GB memory and 2 Cores machine as one of your nodes without changing max to 5GB globally. You can set yarn.nodemanager.resource,memory-mb=20000 on machines with 20gb memory and set yarn.nodemanager.resource,memory-mb=5000 on machine with 5gb memory. You can also manage different configuration on different node managers using ambari. Its called host config groups. https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.1.0/bk_ambari-user-guide/content/using_host_config_groups.html
... View more
04-28-2017
06:29 PM
5 Kudos
Hi Qi Wang: To your question "I know row level filter only has select. But does it also remove the permission from other policy? Like hive users universal access from the base policy.” --> In this case your user 'hive' has all permissions on all tables through the default access based policy. Also there is a row level policy for user 'hive’ on 'rowfilter' table. We have only ‘select’ privilege to choose in row level policy, what this means is that you are giving user ‘hive’ only ‘select' privilege on the table and that too with restrictions on what he can view on doing ‘select’. Why this is done? If you as a user are not allowed to even see the full contents of a particular table, you cannot be allowed to perform operations on that table. When Hive request comes in from a user, all row level policies are scanned for that user. If a row level policy found for the user, the access privileges are scanned on that resource by going through access policies. (An access policy needs to grant the user access to that resource, row filer policy does not grant access). Now since update is not granted by masking policy, the operation is denied. The request does not even reach access policy. Hope this helps.
... View more
12-09-2016
02:47 AM
To confirm, the issue is that hbase conf was not available to spark. You can also check the Spark HBase Connector we support at https://github.com/hortonworks-spark/shc. It has many features but also documents the configuration for Spark Hbase access and security aspects too.
... View more
02-02-2016
02:55 PM
When I added Hive action before the Sqoop action, I got hive-site.xml file permission error. To avoid this problem, delete lib/hive-site.xml file, and add this file to inside of Sqoop action as a file.
... View more
02-03-2016
04:45 PM
awesome! I will try this out and maybe publish an article @cmuchinsky
... View more