Member since
05-02-2019
319
Posts
145
Kudos Received
59
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
7121 | 06-03-2019 09:31 PM | |
1725 | 05-22-2019 02:38 AM | |
2179 | 05-22-2019 02:21 AM | |
1360 | 05-04-2019 08:17 PM | |
1675 | 04-14-2019 12:06 AM |
05-21-2017
10:33 PM
I surely don't have an answer for this one, but you could ~play~ with it by hand-jamming what you think is the appropriate jar into the worker nodes as I did with another jar issue as described in https://martin.atlassian.net/wiki/x/JbXqBQ. Then you'd have an easy case to submit which the support team could easily reproduce (and get fixed!). Good luck and happy Hadooping/Storming.
... View more
05-21-2017
09:05 PM
1 Kudo
When calling setBolt() on the TopologyBuilder you are then using fieldsGrouping() to link it to a particular component/stream? You do agree that you won't get just that combination of deviceid/siteid/ip in each of the bolt instances, but you should be getting all of a unique combination going to the same bolt instance, right? Maybe you could show some code and someone could help you out better as it does sound like you are going down the right path. It would also be interesting (helpful maybe even) to know how many instances of the bolt you have as well as a guesstimate of the distinct number of deviceid/siteid/ip combinations you have.
... View more
04-13-2017
10:39 PM
This is something on the horizon, but not something we are ready to announce a roadmap for.
... View more
04-10-2017
01:43 PM
Looks like a couple of good answers. Please check out https://martin.atlassian.net/wiki/x/AunyBQ and if one of these answer helps you the "best", would you mind "Accepting" it??
... View more
03-28-2017
07:22 PM
We continuing to review the entire certification offerings for if & when changes are to be made. We will be sure to give adequate time to all folks preparing for exams if/when changes are made. As for bundling a HCA voucher in with the Essentials course, no decision has been made to do so as of yet.
... View more
03-08-2017
10:47 PM
Beautiful!! This fixes me!! Especially, when I noticed the storm.yaml goes in a .storm file in the home dir (I didn't create that subdir initially). The only thing I did different is leave the default /etc/storm/conf/client_jaas.conf which reads as the following. StormClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true
renewTicket=true
serviceName="nimbus";
}; 1,000,000 thanks!! A shout out to @Vipin Rathor for the second set of eyes!!
... View more
03-08-2017
07:47 PM
Yep, this worked for me -- from sandbox to mac, but still can't go the other direction. Thanks!
... View more
03-08-2017
03:51 PM
Thanks for all the great responses here and below. Yes, indeed, the worker nodes that I have all of this running on are overloaded. Taking into account the distribution of services and the underlying box resource footprint. Thanks again!!
... View more
03-07-2017
09:20 PM
Running Storm on a Kerberized HDP 2.5.3 cluster and am able to run "storm list" to see no topologies are running and the Storm UI reports my two nimbus servers are running as well as my five supervisors. Now I'm getting the following excerpt when I tried to run the simple WordCount topology described in Step 4 on http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_command-line-installation/content/validate_installation_storm.html. [student2@ip-172-xxx-xxx-42 ~]$ storm jar /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-topologies-*.jar org.apache.storm.starter.WordCountTopology wordcount
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
Running: /usr/java/default/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.5.3.0-37/storm -Dstorm.log.dir= -Djava.library.path= -Dstorm.conf.file= -cp /usr/hdp/2.5.3.0-37/storm/lib/asm-5.0.3.jar:RM'D_LONG_LIST_OF_JARS_FOR_READABILITY org.apache.storm.daemon.ClientJarTransformerRunner /usr/hdp/current/storm-client/contrib/storm-starter/storm-starter-topologies-1.0.1.2.5.3.0-37.jar /tmp/40f0348a037911e7b88f02ad93562fff.jar
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 85983232 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/LAB.HORTONWORKS.NET/student2/hs_err_pid11132.log
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
Running: /usr/java/default/bin/java -client -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.5.3.0-37/storm -Dstorm.log.dir= -Djava.library.path= -Dstorm.conf.file= -cp /usr/hdp/2.5.3.0-37/storm/lib/asm-5.0.3.jar:RM'D_LONG_LIST_OF_JARS_FOR_READABILITY -Dstorm.jar=/tmp/40f0348a037911e7b88f02ad93562fff.jar org.apache.storm.starter.WordCountTopology wordcount
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076e980000, 85983232, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 85983232 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/LAB.HORTONWORKS.NET/student2/hs_err_pid11138.log
Traceback (most recent call last):
File "/usr/hdp/2.5.3.0-37/storm/bin/storm.py", line 774, in <module>
main()
File "/usr/hdp/2.5.3.0-37/storm/bin/storm.py", line 771, in main
(COMMANDS.get(COMMAND, unknown_command))(*ARGS)
File "/usr/hdp/2.5.3.0-37/storm/bin/storm.py", line 248, in jar
os.remove(tmpjar)
OSError: [Errno 2] No such file or directory: '/tmp/40f0348a037911e7b88f02ad93562fff.jar'
[student2@ip-172-xxx-xxx-42 ~]$ The referenced file is available: hs-err-pid11132log.txt (added a .txt so HCC would allow it as an attachment). I tried to increase the memory foot print of Nimbus from 1GB to 2GB and Supervisors from 768MB to 2GB with the changes highlighted below, but same error before and after this change. Did I make the right changes to give these services more memory?? Is there something else I should be trying??
... View more
Labels: