Member since
09-20-2017
50
Posts
2
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2252 | 08-09-2018 06:47 PM | |
6457 | 01-05-2018 02:34 PM | |
1662 | 12-05-2017 02:29 PM | |
987 | 10-18-2017 06:10 PM |
12-12-2017
07:30 PM
I am able to create a policy with the below command; curl -iv -u username:password -H "content-type:application/json" -X POST http://hostname:6080/service/public/api/policy/ -d '{ "policyName": "api-Test", "resourceName": "/data", "description": "Testing", "repositoryName": "HDPPRD01_hadoop", "repositoryType": "hdfs", "isEnabled": "true", "isRecursive": "true", "isAuditEnabled": "true", "permMapList": [{ "userList":["sudheer"],"groupList":["hadoop_group"], "permList": ["Read","Execute", "Write", "Admin"] }] }' But I want to update that policy with the Rest APi: Tried with this command but throwing some error: (The specified HTTP method is not allowed for the requested resource.) curl -iv -u username:password -H "content-type:application/json" -X PUT http://hostname:6080/service/public/api/policy/ -d '{ "policyName": "api-Test", "resourceName": "/tmp", "description": "Testing", "repositoryName": "HDPPRD01_hadoop", "repositoryType": "hdfs", "isEnabled": "true", "isRecursive": "true", "isAuditEnabled": "true", "permMapList": [{ "userList":["velagapudi"],"groupList":["hadoop_user"], "permList": ["Read","Execute", "Write", "Admin"] }] }'
... View more
Labels:
- Labels:
-
Apache Ranger
12-05-2017
02:29 PM
Run the below command to manually replicate the ranger_audits to other solr instance. curl -i -k -v --negotiate -u : http://<from-node>:8886/solr/admin/collections?action=ADDREPLICA&collection=ranger_audits&shard=shard1&node=<target-host>:8886_solr
... View more
11-21-2017
02:57 PM
My Solr instance is getting killed once in a week with OOM. I tried to tune the below parameter.. Need some recommendations.. Recommended values for these parameters based on the log provided: GC_TUNE="-XX:NewRatio=3 \
-XX:SurvivorRatio=4 \
-XX:TargetSurvivorRatio=90 \
-XX:MaxTenuringThreshold=8 \
-XX:+UseConcMarkSweepGC \
-XX:+UseParNewGC \
-XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 \
-XX:+CMSScavengeBeforeRemark \
-XX:PretenureSizeThreshold=64m \
-XX:+UseCMSInitiatingOccupancyOnly \
-XX:CMSInitiatingOccupancyFraction=50 \
-XX:CMSMaxAbortablePrecleanTime=6000 \
-XX:+CMSParallelRemarkEnabled \
-XX:+ParallelRefProcEnabled"
And also:
Minimum Heap Size & Maximum Heap Size Log: Heap after GC invocations=19 (full 1):
par new generation total 218496K, used 7215K [0x00000006c0000000, 0x00000006d0000000, 0x0000000700000000)
eden space 174848K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x00000006caac0000)
from space 43648K, 16% used [0x00000006cd560000, 0x00000006cdc6bc78, 0x00000006d0000000)
to space 43648K, 0% used [0x00000006caac0000, 0x00000006caac0000, 0x00000006cd560000)
concurrent mark-sweep generation total 786432K, used 92764K [0x0000000700000000, 0x0000000730000000, 0x00000007c0000000)
Metaspace used 39260K, capacity 39748K, committed 40076K, reserved 1085440K
class space used 4267K, capacity 4422K, committed 4528K, reserved 1048576K
}
2017-11-21T14:30:19.916-0500: 117.112: Total time for which application threads were stopped: 0.0098877 seconds, Stopping threads took: 0.0000449 seconds
2017-11-21T14:30:20.916-0500: 118.112: Total time for which application threads were stopped: 0.0003058 seconds, Stopping threads took: 0.0001195 seconds
{Heap before GC invocations=19 (full 1):
par new generation total 218496K, used 182063K [0x00000006c0000000, 0x00000006d0000000, 0x0000000700000000)
eden space 174848K, 100% used [0x00000006c0000000, 0x00000006caac0000, 0x00000006caac0000)
from space 43648K, 16% used [0x00000006cd560000, 0x00000006cdc6bc78, 0x00000006d0000000)
to space 43648K, 0% used [0x00000006caac0000, 0x00000006caac0000, 0x00000006cd560000)
concurrent mark-sweep generation total 786432K, used 92764K [0x0000000700000000, 0x0000000730000000, 0x00000007c0000000)
Metaspace used 39262K, capacity 39748K, committed 40076K, reserved 1085440K
class space used 4267K, capacity 4422K, committed 4528K, reserved 1048576K
2017-11-21T14:30:28.151-0500: 125.346: [GC (Allocation Failure) 2017-11-21T14:30:28.151-0500: 125.346: [ParNew
Desired survivor size 40225992 bytes, new threshold 8 (max 8)
- age 1: 136184 bytes, 136184 total
- age 2: 1010336 bytes, 1146520 total
- age 3: 40472 bytes, 1186992 total
- age 4: 73744 bytes, 1260736 total
- age 5: 2934424 bytes, 4195160 total
- age 6: 24424 bytes, 4219584 total
- age 7: 111680 bytes, 4331264 total
- age 8: 56160 bytes, 4387424 total
: 182063K->5404K(218496K), 0.0065729 secs] 274827K->98648K(1004928K), 0.0066500 secs] [Times: user=0.02 sys=0.00, real=0.01 secs]
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Solr
11-17-2017
06:05 PM
And also enable yarn acl in yarn>config>Resource Manager yarn.acl.enable=true
yarn.admin.acl=<add the user to this list>
... View more
11-13-2017
06:41 PM
How to replicate my ranger audits in ambari-infra to 3 instances from 1 which I have now.. I tried to change the value of ranger_solr_replication_factor value to 3 and restarted ranger and ambari-infra but still it is not replicated...
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Ranger
-
Apache Solr
10-18-2017
06:10 PM
Restart the zookeeper and stop & start one of the Resource manager.. It will automatically put one in standby and the other in active.
... View more
10-17-2017
06:51 PM
How can I comment out some of the lines in my code or (add comments in my program) in zeppelin %jdbc(hive) interpreter? I have tried --, // , /*, etc but that didn't work.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Zeppelin
09-22-2017
03:24 PM
Unable to query hive acid with
spark getting following exception Caused by:
java.util.concurrent.ExecutionException: java.lang.NumberFormatException: For
input string: "0012050_0000" at
java.util.concurrent.FutureTask.report(FutureTask.java:122) at
java.util.concurrent.FutureTask.get(FutureTask.java:192) at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:998) ... 118
more Caused by:
java.lang.NumberFormatException: For input string:
"0012050_0000" at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at
java.lang.Long.parseLong(Long.java:589) at
java.lang.Long.parseLong(Long.java:631) at
org.apache.hadoop.hive.ql.io.AcidUtils.parseDelta(AcidUtils.java:310) at
org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:379) at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:634) at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$FileGenerator.call(OrcInputFormat.java:620) at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
... View more
Labels:
- Labels:
-
Apache Spark
09-22-2017
03:13 PM
Your files are owned by UID's instead of usernames.. May be your unix team has changed the UID's of service users.. Change the ownership of files owned by 37990 to 'hdfs' user by using chown linux command..
... View more
- « Previous
- Next »