Member since
03-14-2016
4721
Posts
1111
Kudos Received
874
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2452 | 04-27-2020 03:48 AM | |
4888 | 04-26-2020 06:18 PM | |
3977 | 04-26-2020 06:05 PM | |
3221 | 04-13-2020 08:53 PM | |
4928 | 03-31-2020 02:10 AM |
03-08-2017
05:08 AM
@Lester Martin Also as per the attached "hs_err_pid" file we see that you have only 161 MB of memory left out of 16 GB. Memory: 4k page, physical 16004820k(161604k free), swap 0k(0k free) . So try to either increase the RAM or stop unwanted components/processes that are consuming more memory on your OS.
... View more
03-08-2017
04:56 AM
@Lester Martin If you see that the storm process is getting crashed even though you have enough memory (swap/free) available then you should also check the "/proc/sys/vm/overcommit_memory" - This switch knows 3 different settings: => 0: The Linux kernel is free to over commit memory(this is the default), a heuristic algorithm is applied to figure out if enough memory is available. => 1: The Linux kernel will always over commit memory, and never check if enough memory is available. This increases the risk of out-of-memory situations, but also improves memory-intensive workloads. => 2: The Linux kernel will not over commit memory, and only allocate as much memory as defined in over commit_ratio. As sometimes OS kills /crashes a process due to a system OS setting, the system OS memory overcommit setting was 2 (when it should have been set to 0)
- https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-captun.html
... View more
03-08-2017
04:46 AM
1 Kudo
@Lester Martin You are not getting "OutOfMemory in Java Heap", but it is actually "Native Out Of Memory" Out of Memory Error (os_linux.cpp:2638) . So increasing the Xmx (Heap Memory) will make the issue more worse here. I will suggest you to reduce heap (Xmx) also please check the output of `free -m` before starting the storm process to findout if the OS really has enough memory to allocate? It will also help us in knowing if already swap memory was being used? free -m .
... View more
03-07-2017
12:56 PM
Linux commands: To check memory # free -m To check if CPU resources are enough # top Else try to stop unwanted services that are running on your sandbox to free some resources.
... View more
03-07-2017
12:47 PM
1 Kudo
@voca voca How much free memory do you have remaining in your Sandbox VM? Can you increase the RAM and then try.
... View more
03-07-2017
07:36 AM
1 Kudo
@Devpriyo Bhattacharya Edit the "/etc/ambari-server/conf/ambari.properties" file and lower the value as following. (You seem to have defined too large value there "900000000000000") server.connection.max.idle.millis=900000 . The default value can be seen (as '900000') in the Ambari code base as following: https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java#L2005-L2007 . That seems to be causing the following error: ERROR [main] AmbariServer:844 - Failed to run the Ambari Server java.lang.NumberFormatException: For input string: "900000000000000"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
... View more
03-07-2017
06:58 AM
@Namit Maheshwari
Can you try from some other web browser? Or try it using Incognito mode of the browser once Or logout and login again and then try.
... View more
03-07-2017
06:46 AM
1 Kudo
@Namit Maheshwari Sometime it happens when we try to very quickly edit the same article multiple times. May be you can retry it again in some time to see if it works. Right now i am able to edit my articles.
... View more
03-06-2017
02:30 PM
@shadi Balloul As you mentioned that "There are no logs in the log directory." That could be because of missing "${zookeeper.log.dir}" prefix. Please check if you have added the "${zookeeper.log.dir}" to zookeeper log file as following in your ambari config. Ambari UI --> "Zookeeper" --> "Configs" --> Advanced --> "Advanced zookeeper-log4j"
. CHANGE from: log4j.appender.ROLLINGFILE.File=zookeeper.log
TO Following: log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/zookeeper.log . After making the changes restart the zookeeper service and then please share the log "${zookeeper.log.dir}/zookeeper.log" .
... View more
03-06-2017
09:33 AM
@Krishna Srinivas
The spark-assembly jar can be found in the following location: # ls -l /usr/hdp/2.5.3.0-37/spark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-37.jar
-rw-r--r--. 1 root root 188897932 Nov 30 04:01 /usr/hdp/2.5.3.0-37/spark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-37.jar
# Which is a symlink
# ls -lnrt /usr/hdp/2.5.3.0-37/spark/lib/spark-hdp-assembly.jar
lrwxrwxrwx. 1 0 0 88 Jan 25 07:16 /usr/hdp/2.5.3.0-37/spark/lib/spark-hdp-assembly.jar -> /usr/hdp/2.5.3.0-37/spark/lib/spark-assembly-1.6.2.2.5.3.0-37-hadoop2.7.3.2.5.3.0-37.jar
. However the next error that you are getting as following indicates that you have multiple version of Spark JARs present in your classpath. Can you please remove the other version of spark jars and then try? java.lang.NoSuchMethodError: org.apache.spark.sql.SQLContext.createDataFrame(Lorg/apache/spark/rdd/RDD;Lorg/apache/spark/sql/types/StructType;)Lorg/apache/spark/sql/Dataset; .
... View more