Member since
02-08-2016
793
Posts
669
Kudos Received
85
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2517 | 06-30-2017 05:30 PM | |
3248 | 06-30-2017 02:57 PM | |
2708 | 05-30-2017 07:00 AM | |
3155 | 01-20-2017 10:18 AM | |
6374 | 01-11-2017 02:11 PM |
01-11-2017
02:11 PM
1 Kudo
@Dinesh Das Yes. You can achieve this using ranger. Please check - http://hortonworks.com/blog/best-practices-for-hive-authorization-using-apache-ranger-in-hdp-2-2/
... View more
01-11-2017
09:15 AM
@Zeev Lazarev Can you access the machine directly from console? Pls try doing so. Also if the ip is changed for the machine please check ipaddress and try using ipadress instead of 127.0.0.1 in ssh command.
... View more
01-08-2017
12:17 PM
@Lawrence Lau
HDP source RPM are available on Hortonworks GITHUB and not sure if that is accessible publicly.
... View more
01-06-2017
12:43 PM
Problem Statement: Whenever HBase is running, the "hdfs fsck /" reports four hbase-related files in the path "hbase/data/WALs/" as CORRUPT. Even after letting the cluster sit idle for a couple hours, it is still in the corrupt state. If HBase is shut down, the problem goes away. If HBase is then restarted, the problem recurs. ERROR: hdfs@test1:>$ hdfs fsck /
Connecting to namenode via http://test1.example.com:50070/fsck?ugi=hdfs&path=%2F
FSCK started by hdfs (auth:SIMPLE) from /39.0.8.2 for path / at Wed Jun 24 20:40:17 GMT 2015
...
/apps/hbase/data/WALs/test2.example.com,16020,1435168292684/test2.example.com%2C16020%2C1435168292684.default.1435175500556: MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/test3.example.com,16020,1435168290466/test3.example.com%2C16020%2C1435168290466..meta.1435175562144.meta: MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/test3.example.com
,16020,1435168290466/test3.example.com%2C16020%2C1435168290466.default.1435175498500: MISSING 1 blocks of total size 83 B.
/apps/hbase/data/WALs/est4.example.com,16020,1435168292373/test4.example.com%2C16020%2C1435168292373.default.1435175500301: MISSING 1 blocks of total size 83 B..................................................................................................
....................................................................................................
....................................................................................................
........................................................................................Status: CORRUPT
Total size: 723977553 B (Total open files size: 332 B)
Total dirs: 79
Total files: 388
Total symlinks: 0 (Files currently being written: 5)
Total blocks (validated): 387 (avg. block size 1870743 B) (Total open file blocks (not validated): 4)
********************************
UNDER MIN REPL'D BLOCKS: 4 (1.0335917 %)
dfs.namenode.replication.min: 1
CORRUPT FILES: 4
MISSING BLOCKS: 4
MISSING SIZE: 332 B
********************************
Minimally replicated blocks: 387 (100.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 3.0
Corrupt blocks: 0
Missing replicas: 0 (0.0 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Wed Jun 24 20:40:17 GMT 2015 in 7 milliseconds
The filesystem under path '/' is CORRUPT
hdfs@test1:>
ROOT CAUSE: This is a BUG with HDFS - https://issues.apache.org/jira/browse/HDFS-8809 There is a patch provided for resolving this issue. This will be soon fixed in upcoming HDP releases.
... View more
Labels:
01-04-2017
08:41 AM
2 Kudos
Below article will guide you on how to access Ranger KMS policies via rest api - 1. I have Cluster with Ranger and Ranger KMS installed. 2. From the documentation it is clearly given on "how to access Ranger policies using rest api". Please check the link below - https://cwiki.apache.org/confluence/display/RANGER/REST+APIs+for+Policy+Management 3. Below is the example for Range rest api GET method - a.Sample Ranger Policy for HDFS repository in UI is as below - Below is the rest api GET method call which we can use to get the policy as displayed above - curl -iv -u <username>:<password> -H "Content-type:application/json" -X GET http://localhost:6080/service/public/api/repository/{id}
Eg. In above screenshot my policy id is "2"
curl -iv -u admin:admin -H "Content-type:application/json" -X GET http://localhost:6080/service/public/api/repository/2
4. Below is my Ranger KMS UI with policy - But if you try the same steps for Ranger KMS it will fail - curl -iv -u <username>:<password> -H "Content-type:application/json" -X GET http://localhost:6080/service/public/api/repository/{id}
Eg. In above screenshot my policy id is "1"
curl -iv -u admin:admin -H "Content-type:application/json" -X GET http://localhost:6080/service/public/api/repository/1 ERROR: [root@localhost ~]# curl -iv -u admin:admin -H "Content-type:application/json" -X GET http://localhost:6080/service/public/api/repository/1
* About to connect() to khichadi1.openstacklocal port 6080 (#0)
* Trying 172.26.81.49... connected
* Connected to khichadi1.openstacklocal (172.26.81.49) port 6080 (#0)
* Server auth using Basic with user 'admin'
> GET /service/public/api/repository/1 HTTP/1.1
> Authorization: Basic YWRtaW46YWRtaW4=
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.16.1 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: khichadi1.openstacklocal:6080
> Accept: */*
> Content-type:application/json
>
< HTTP/1.1 204 No Content
HTTP/1.1 204 No Content
< Server: Apache-Coyote/1.1
Server: Apache-Coyote/1.1
< Set-Cookie: JSESSIONID=30095A2BD0FBFB384F24734183851135; Path=/; HttpOnly
Set-Cookie: JSESSIONID=30095A2BD0FBFB384F24734183851135; Path=/; HttpOnly
< X-Frame-Options: DENY
X-Frame-Options: DENY
< Content-Type: application/json
Content-Type: application/json
< Date: Wed, 04 Jan 2017 08:32:57 GMT
Date: Wed, 04 Jan 2017 08:32:57 GMT
<
* Connection #0 to host khichadi1.openstacklocal left intact
* Closing connection #0
5. Since the Rest api for Ranger and Ranger KMS is little bit different. Below is how it works - In above examples instead of /service/public/api/repository/{id} you need to use /service/plugins/policies/{id} for Ranger KMS 6. Below is the sample example for GET method for Ranger KMS - curl -iv -u <username>:<password> -H "Content-type:application/json" -X GET http://localhost:6080/service/plugins/policies/{id}
Eg. In above screenshot my policy id is "1"
curl -iv -u admin:admin -H "Content-type:application/json" -X GET http://localhost:6080/service/plugins/policies/1 For rest of the method like create/update/delete you can use above examples replacing the method type. Refer example for details on - https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.6+-+REST+APIs+for+Service+Definition%2C+Service+and+Policy+Management Do modify accordingly for Ranger KMS rest api for above examples in link. Let me know if you have any questions for above article. Thanks.
... View more
Labels:
01-04-2017
06:10 AM
@Ali Nazemian
I see this is a BUG Please check the link - https://issues.apache.org/jira/browse/AMBARI-2644 Below is the HWX bug url - https://hortonworks.jira.com/browse/BUG-7301 There is Patch provided to fix this issue.
... View more
01-03-2017
04:35 PM
Also is sugu ldap/AD user? or local user? Can you check the uid and gid for the user on all nodes and confirm if its same across all nodes?
... View more
01-03-2017
04:27 PM
1 Kudo
@Sami Ahmad Can you make sure if the user is present on all nodes in the cluster? Suspected that the application where its running on the node the user sugu is not present.
... View more