Member since
04-09-2019
254
Posts
140
Kudos Received
34
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2089 | 05-22-2018 08:32 PM | |
14282 | 03-15-2018 02:28 AM | |
3802 | 08-07-2017 07:23 PM | |
4596 | 07-27-2017 05:22 PM | |
2595 | 07-27-2017 05:16 PM |
06-03-2017
08:14 PM
Pretty informative and useful. Thanks @Dominika Bialek for writing this. Keep it up !!
... View more
06-02-2017
01:52 AM
4 Kudos
Unlike other services, Knox doesn't expose Java Heap settings via Ambari. Follow these steps to change default Heap settings for Knox: 1. On the Knox node, login as root and go to this directory: /usr/hdp/current/knox-server/bin # cd /usr/hdp/current/knox-server/bin 2. Make a copy of the file that we are going to change next. # cp gateway.sh gateway.sh.backup.`date +%m%d%Y-%H-%M-%S` 3. Open gateway.sh in text editor and change this line: APP_MEM_OPTS=""
to this:
APP_MEM_OPTS="-Xmx5g -XX:NewSize=3G -XX:MaxNewSize=3G -verbose:gc -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -Xloggc:/var/log/knox/knox-gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps"
Shown above are the example values for Knox Heap. This should be updated based on user's environment. General formula for Knox Heap size will be: Recommended KNOX Heap Size =
(Webhdfs(ConcNumOfRequest*(replayBufferSize)) +
HBase(ConcNumOfRequest*(replayBufferSize)) +
Hive(ConcNumOfRequest*(replayBufferSize))) + 20 % for request surges where: ConcNumOfRequest = Number of concurrent requests expected for each component (WebHDFS, HBase, Hive etc.) replayBufferSize = Size of the largest incoming request to Knox Based on the Heap size value, user need to further tune the NewSize and MaxNewSize. 4. Save changes to gateway.sh and restart Knox via Ambari. Confirm the new settings via 'ps -ef | grep knox' command. # ps -ef| grep gateway
knox 29236 1 93 23:18 ? 00:00:29 /usr/jdk64/jdk1.8.0_77/bin/java -Xmx5g -XX:NewSize=3G -XX:MaxNewSize=3G -verbose:gc -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -Xloggc:/var/log/knox/knox-gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -jar /usr/hdp/current/knox-server/bin/gateway.jar
... View more
Labels:
11-16-2017
08:24 PM
First of all, Nic Swart's comment is VERY IMPORTANT! Thanks much. Without the extra parentheses it just doesn't work (doesn't bother to tell you that the syntax of the ldap query is wrong, grrr!) Second, To be clear, the ldapRealm.xxx settings are recommended for use with Active Directory starting with Zeppelin 0.7. This can be a little confusing.
... View more
10-24-2018
09:19 PM
how can i see the output of hive command with job_id. Also can you please help me to understand the command which will have query as well as query output stored in a file with single command
... View more
05-10-2017
06:29 AM
Hey @Timothy Spann, this is a really cool demo involving all my favorites Hadoop, Rpi, NiFi and IoT. Great job & Keep it up!
... View more
04-27-2017
04:57 PM
Nice work @Arpit Khare ! This is going to be quite useful for the folks around. Thank you and keep it up !!
... View more
10-09-2018
01:35 PM
It is recommended to disable the cron in zeppelin as there is no validation if the user is allowed to run as the hive/hdfs or any other user specified in the cron field, meaning any user can set it to be run as any user. disbale is possible from 2.6.5, for previous versiosn please engage the hortonworks support.
... View more
04-25-2017
03:30 PM
1 Kudo
Thanks all for your responses. Once again i reassign ownership. It works!!! ## hdfs dfs -chown -R admin:hadoop /user/admin
... View more
04-01-2017
03:12 AM
2 Kudos
@Vipin Rathor
Wonderful article!!! I was facing this issue for quite sometime. I will implement the fix to see how it goes.
... View more
03-30-2017
05:54 PM
1 Kudo
yes surya it was one way ssl
... View more