Member since
04-25-2016
579
Posts
609
Kudos Received
111
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1986 | 02-12-2020 03:17 PM | |
1370 | 08-10-2017 09:42 AM | |
10124 | 07-28-2017 03:57 AM | |
2154 | 07-19-2017 02:43 AM | |
1669 | 07-13-2017 11:42 AM |
11-15-2016
03:37 PM
2 Kudos
can you try this query using http mode by setting hive.server2.transport.mode to http instead of binary. do not forget to restart HS2.
... View more
11-15-2016
01:30 PM
2 Kudos
no it will occupy 3*10 = 30M on hdfs,HDFS capacity is consumed based on the actual file size but a block is consumed per file
... View more
11-15-2016
07:29 AM
1 Kudo
look for Configuring Users & Access Policies in https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html
... View more
11-15-2016
07:17 AM
looks you have secure env, the reason of NPE could be accessiblity of your zknode, how you are accessing zk node.
... View more
11-15-2016
06:11 AM
3 Kudos
/hbase-secure is not hdfs path it is zk node, please check this zk node using zkCli
... View more
11-14-2016
10:59 AM
@Gobi Subramani in case of memory pressure Spark will automatically evict RDD partitions from Workers in an LRU manner if no caching or persistence applied. depending on worker memory available LRU eviction happens independently on each Worker node.
... View more
11-14-2016
10:08 AM
2 Kudos
@Gobi Subramani spark keeps lineage for all the previous RDD in order to calculate the next RDD, not sure what the purge mean here but if there is no memory available then spark with throw oom and fail the job. as OOM is error so previous RDD is not recoverable in this case.
... View more
10-18-2016
11:08 PM
1 Kudo
HiveServer2 does not expose jmx or equivalent metrics but often we require to monitor the Hiveserver2 process to know cpu usage, heap usage and state of various thread along with some other params like no of class loaded by hs2 jvm. it is also very useful if you want to run some cpu/memory sampling to profile hiveserver2. Step 1: login into Ambari and open hive advanced configuration Step 2: expand advanced hive-env settings and add the following configuration in hive-env template if [ "$SERVICE" = "hiveserver2" ]; then
export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=8008"
fi
Step 3: Restart HiveServer2 Step 4: on client machine open jvisualvm($JAVA_HOME/bin), once open go to File -> Add JMX Connection -> enter hostname and port no(it's 8008 in this case) and you are good to go.
... View more
Labels:
10-16-2016
05:14 AM
@Davd Leh good to hear that it helped, could you accept a best answer here so that other user can get benefit of it.thanks
... View more