Thanks for your response.If I partition the data by yyyy-mm-dd field and I receive only one file per day. I assume , I will always have one file per partition irrespective of this setting? If the above assumption is correct(pls correct if that is wrong), will I end up with select queries which runs slower if I store files for say 6 years? i.e I will have 6 * 365 files each around say 8-9MB in size (which is smaller than the default chunk size). I was hoping to consolidate the files on a weekly basis but , I need to have data available to users on a daily basis ..hence I dont think I can do that. Let me know the suggestions. Thanks Nikkie ... View more

Hi, I think I figured it out myself. Here is what I did curl -u admin:admin -X GET http://127.0.0.1:6080/service/public/v2/api/servicedef/name/hive >> test.out Opened the test.out file. The mistake I did previously was I sent only the policyconditions in the file. Now the full output from the GET with the udpated policycondition is sent back. Updated the policycondition section which was blank in the response of GET as below(bold) { "id": 3, "guid": "3e1afb5a-184a-4e82-9d9c-87a5cacc243c", "isEnabled": true, "createTime": 1477381370000, "updateTime": 1477381412000, "version": 2, "name": "hive", "implClass": "org.apache.ranger.services.hive.RangerServiceHive", "label": "Hive Server2", "description": "Hive Server2", "options": {}, "configs": [ { "itemId": 1, "name": "username", "type": "string", "mandatory": true, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Username" }, { "itemId": 2, "name": "password", "type": "password", "mandatory": true, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Password" }, { "itemId": 3, "name": "jdbc.driverClassName", "type": "string", "mandatory": true, "defaultValue": "org.apache.hive.jdbc.HiveDriver", "validationRegEx": "", "validationMessage": "", "uiHint": "" }, { "itemId": 4, "name": "jdbc.url", "type": "string", "mandatory": true, "defaultValue": "", "validationRegEx": "", "validationMessage": "", "uiHint": "" }, { "itemId": 5, "name": "commonNameForCertificate", "type": "string", "mandatory": false, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Common Name for Certificate" } ], "resources": [ { "itemId": 1, "name": "database", "type": "string", "level": 10, "mandatory": true, "lookupSupported": true, "recursiveSupported": false, "excludesSupported": true, "matcher": "org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher", "matcherOptions": { "wildCard": "true", "ignoreCase": "true" }, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Hive Database", "description": "Hive Database" }, { "itemId": 2, "name": "table", "type": "string", "level": 20, "parent": "database", "mandatory": true, "lookupSupported": true, "recursiveSupported": false, "excludesSupported": true, "matcher": "org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher", "matcherOptions": { "wildCard": "true", "ignoreCase": "true" }, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Hive Table", "description": "Hive Table" }, { "itemId": 3, "name": "udf", "type": "string", "level": 20, "parent": "database", "mandatory": true, "lookupSupported": true, "recursiveSupported": false, "excludesSupported": true, "matcher": "org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher", "matcherOptions": { "wildCard": "true", "ignoreCase": "true" }, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Hive UDF", "description": "Hive UDF" }, { "itemId": 4, "name": "column", "type": "string", "level": 30, "parent": "table", "mandatory": true, "lookupSupported": true, "recursiveSupported": false, "excludesSupported": true, "matcher": "org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher", "matcherOptions": { "wildCard": "true", "ignoreCase": "true" }, "validationRegEx": "", "validationMessage": "", "uiHint": "", "label": "Hive Column", "description": "Hive Column" } ], "accessTypes": [ { "itemId": 1, "name": "select", "label": "select", "impliedGrants": [] }, { "itemId": 2, "name": "update", "label": "update", "impliedGrants": [] }, { "itemId": 3, "name": "create", "label": "Create", "impliedGrants": [] }, { "itemId": 4, "name": "drop", "label": "Drop", "impliedGrants": [] }, { "itemId": 5, "name": "alter", "label": "Alter", "impliedGrants": [] }, { "itemId": 6, "name": "index", "label": "Index", "impliedGrants": [] }, { "itemId": 7, "name": "lock", "label": "Lock", "impliedGrants": [] }, { "itemId": 8, "name": "all", "label": "All", "impliedGrants": [ "select", "update", "create", "drop", "alter", "index", "lock" ] } ], "policyConditions": [ { "itemId": 1, "name": "resources-accessed-together", "evaluator": "org.apache.ranger.plugin.conditionevaluator.RangerHiveResourcesAccessedTogetherCondition", "evaluatorOptions": {}, "label": "Resources Accessed Together?", "description": "Resources Accessed Together?" }], "contextEnrichers": [], <Deleted the remaninig> ] } } Saved the file as hiveService3.json Executed the below command. curl -v -H 'Content-Type: application/json' -u admin:admin -X PUT --data @hiveService3.json http://127.0.0.1:6080/service/public/v2/api/servicedef/name/hive * About to connect() to 127.0.0.1 port 6080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 6080 (#0) * Server auth using Basic with user 'admin' > PUT /service/public/v2/api/servicedef/name/hive HTTP/1.1 > Authorization: Basic YWRtaW46YWRtaW4= > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: 127.0.0.1:6080 > Accept: */* > Content-Type: application/json > Content-Length: 10161 > Expect: 100-continue > < HTTP/1.1 100 Continue < HTTP/1.1 200 OK < Server: Apache-Coyote/1.1 < Set-Cookie: RANGERADMINSESSIONID=E0EA0005D86487C03AB4A3C2129E3A97; Path=/; HttpOnly < X-Frame-Options: DENY < Content-Type: application/json < Transfer-Encoding: chunked < Date: Tue, 23 May 2017 12:05:34 GMT < Connection #0 to host 127.0.0.1 left intact * Closing connection #0 It seems to be success as when I clicked on the Add Condition from Ranger the condition ResourcesAccessedTogether Came?: I still have to proceed with the next steps. I will let you know. Thanks Nikkie ... View more

Hi, I was trying to work through the example in HDP 2.5 sandbox. I created a file named hiveService2.json with the following content. "policyConditions": [ { "itemId": 1, "name": "resources-accessed-together", "evaluator": "org.apache.ranger.plugin.conditionevaluator.RangerHiveResourcesAccessedTogetherCondition", "evaluatorOptions": {}, "label": "Resources Accessed Together?", "description": "Resources Accessed Together?" } ] Connected to the sandbox via putty. The file is placed at /root/hiveService2.json Executed the following command from /root curl -v -H 'Content-Type: application/json' -u admin:admin -X PUT --data @hiveService2.json http://127.0.0.1:6080/service/public/v2/api/servicedef/name/hive I am getting the below About to connect() to 127.0.0.1 port 6080 (#0) * Trying 127.0.0.1... connected * Connected to 127.0.0.1 (127.0.0.1) port 6080 (#0) * Server auth using Basic with user 'admin' > PUT /service/public/v2/api/servicedef/name/hive HTTP/1.1 > Authorization: Basic YWRtaW46YWRtaW4= > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: 127.0.0.1:6080 > Accept: */* > Content-Type: application/json > Content-Length: 325 > < HTTP/1.1 404 Not Found < Server: Apache-Coyote/1.1 < Set-Cookie: RANGERADMINSESSIONID=4ABEDBD3646557C69F985A11BF7DDE19; Path=/; HttpOnly < X-Frame-Options: DENY < Content-Length: 0 < Date: Tue, 23 May 2017 11:30:40 GMT < * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 Could you please help me 1)Correct the content of the file(hiveService2.json)- if it is not correct 2)Get around the 404 Not Found. Thanks Nikkie ... View more

Ok.. so my understanding was correct. We wouldn't need to have R installed on all the cluster nodes. The client node with RStudioServer only needs it(and it will definitely have it) ? ... View more

@Divakar Annapureddy Thanks for the very quick response. I will try that out. Do you have any insights into the question 1 in the orginal post? ... View more

@Divakar Annapureddy Thanks for your response. We are using the licensed version RStudioServer Pro. Can you throw some light on the questions posted? ... View more