Member since
08-15-2016
189
Posts
63
Kudos Received
22
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
5672 | 01-02-2018 09:11 AM | |
3014 | 12-04-2017 11:37 AM | |
2152 | 10-03-2017 11:52 AM | |
21583 | 09-20-2017 09:35 PM | |
1606 | 09-12-2017 06:50 PM |
10-18-2016
10:37 AM
For the moment I will not use this exclude switch because it behaves not as I (and my client) would expect. I will go for the Deny Conditions extension for the Hive service. The exclude switch is confusing in that it seems to swap an allow into a deny, but it doesn't. It only excludes the resources from the policy
... View more
09-26-2017
08:54 PM
1 Kudo
This procedure has been replaced by https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/about_ranger_policies.html#enable_deny_conditions_for_policies . It is located on a page about implementing tag-based policies. ranger enableDenyAndExceptionsInPolicies=true deny and exception.
... View more
10-12-2016
04:38 PM
1 Kudo
@Jasper Ranger service - configuration has details on which hadoop components is using Ranger for authorization and what policies are there which can be enabled / disabled / audited or not. These service definitions gets created when you enable ranger for the respective components from Ambari and restart the service These properties which are there against the services are used only for Test Connection and Lookup functionality which allows you to select the resource when you maintain policies, i.e auto population of the resource based on the resource name you are going to type in the field. e.g For Hive when you maintain policies if the "Database" field if there are multiple databases in hive and some of them starts with letter "d" and you type "d" in that field in brings up a list of databases which starts with "d". Same case with HDFS it will bring the matching directories for the "PATH". Same with other components too. To do this operation, Ranger communicates with respective hadoop component and brings those details. The user and config maintained here will be used for this communication and in kerberos it will be a service principal which will be used. These users will have policy to do these operations. This is the main purpose of these configuration and it DOESN'T stop you from using the RANGER plugin if the TEST CONNECTION / LOOKUP is not working. It is just for added convenience when maintaining policies. There are lot of misconception around it. In Kerberos environment in HDP 2.5 where ranger itself is kerberized , there are some "Add New Configuration" parameters get configured which maintains various users which communicates with Ranger admin to download policies, tags, service creation from ambari, service check etc.
... View more
10-10-2016
05:48 PM
1 Kudo
Turns out Knox can not be set up easily when you have installed just the local MIT KDC for kerberizing the cluster as per this doc . Or you would have to at least install additional services on top like @Predrag Minovic mentioned. That is the very reason why Knox ships with the example demo LDAP that you can start at the Ambari Knox service config Service Actions > Start Demo LDAP.
... View more
09-28-2016
12:52 PM
That was it.I still had to apply the following to make it work for YARN as well: First generate a secret key and push it to all nodes. Instructions here Then add to custom core-site.xml: hadoop.http.authentication.simple.anonymous.allowed=false
hadoop.http.authentication.signature.secret.file=/etc/security/http_secret
hadoop.http.authentication.type=kerberos
hadoop.http.authentication.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab
hadoop.http.authentication.kerberos.principal=HTTP/_HOST@LAB.HORTONWORKS.NET
hadoop.http.authentication.cookie.domain=lab.hortonworks.net
hadoop.http.filter.initializers=org.apache.hadoop.security.AuthenticationFilterInitializer
Restart ambari-server
... View more
09-25-2016
02:36 PM
OK got it now, The restriction was on the Docker host service level. Just shift the Memory slider and then you should be fine.
... View more
06-27-2019
06:54 PM
Hello All , Newbie and learning. sandbox seems to stuck at the following . 00:00:09.966414 GIM: KVM: VCPU 0: Enabled system-time struct. at 0x0000000002722000 - u32TscScale=0xcd20cfc1 i8TscShift=-1 uVersion=2 fFlags=0x1 uTsc=0x2a3bff811 uVirtNanoTS=0x10eb9c43d 00:00:09.966475 TM: Host/VM is not suitable for using TSC mode 'RealTscOffset', request to change TSC mode ignored 00:00:12.234594 GIM: KVM: Enabled wall-clock struct. at 0x0000000002721008 - u32Sec=1561631780 u32Nano=946977500 uVersion=2 00:00:12.238143 GUI: UIFrameBufferPrivate::performResize: Size=720x400, Using fallback buffer since no source bitmap is provided 00:00:12.363769 PIT: mode=2 count=0x4a9 (1193) - 1000.15 Hz (ch=0) 00:31:50.924499 NAT: Old socket recv size: 64KB 00:31:50.924620 NAT: Old socket send size: 64KB Also, there seems to be a following error 00:00:04.874822 NAT: Failed to redirect TCP 127.0.0.1:8005 -> 0.0.0.0:8005 (Unknown error) tried to open 127.0.0.1 and no luck please advise. have 16gm on a windows 10 machine .
... View more
09-22-2016
09:45 AM
Hi @Timothy Spann and @Jasper , I found the cause of issue now. The issue was I was not putting colon (: ) between port(2181) and hbase tablespace(hbase-unsecure) in spark-shell properly while loading the table. - Earlier I was loading the table in spark-shell as below, which was giving me no Table found error. val jdbcDF = sqlContext.read.format("jdbc").options( Map( "driver" -> "org.apache.phoenix.jdbc.PhoenixDriver", "url" -> "jdbc:phoenix:<host>:2181/hbase-unsecure", "dbtable" -> "TEST_TABLE2") ).load() - But now after putting colon ( : ) between port(2181) number andhbase tablespace (hbase-unsecure). I am able to load table. val jdbcDF = sqlContext.read.format("jdbc").options( Map( "driver" -> "org.apache.phoenix.jdbc.PhoenixDriver", "url" -> "jdbc:phoenix:<host>:2181:/hbase-unsecure", "dbtable" -> "TEST_TABLE2") ).load()
... View more
09-19-2016
05:20 PM
2 Kudos
@srinivasa rao I guess you read about when you perform a "select * from <tablename>", Hive fetches the whole data from file as a FetchTask rather than a mapreduce task which just dumps the data as it is without doing anything on it, similar to "hadoop dfs -text <filename>" However, the above does not take advantage of the true parallelism. In your case, for 1 GB will not make the difference, but image a 100 TB table and you do use a single threaded task in a cluster with 1000 nodes. FetchTask is not a good use of parallelism. Tez provides some options to split the data set to allow true parallelism. tez.grouping.max-size and tez.grouping.min-size are split parameters. Ref: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_installing_manually_book/content/ref-ffec9e6b-41f4-47de-b5cd-1403b4c4a7c8.1.html If any of the responses was helpful, please don't forget to vote/accept the answer.
... View more
- « Previous
- Next »