Member since
07-06-2017
8
Posts
0
Kudos Received
0
Solutions
10-08-2018
01:05 PM
This works indeed. With your example I recognized that my problem was not to provide the parameters to mapreduce.map.java.opts. I supplied the complete opts string with a variable: export MAPPER_OPTS="-Dmapreduce.map.java.opts='-Xss5M -agentlib:jdwp=transport=dt_socket,server=y,address=8787'" (I tried different combinations of singlequote, double quote and masking with backslash) Resolving this in my yarn command caused the problems. (Though having this in a variable because it changes often would be easier. 🙂 ) Thanks for your help, Eddie
... View more
10-08-2018
08:55 AM
Putting the agentlib in YARN_OPTS instead of mapreduce.map.java.opts didn't work for me, debugging is not possible. I also have other use cases where it would be nice to be able to add more properties to mapreduce.map.java.opts. Thanks, Eddie
... View more
10-02-2018
01:59 PM
I am trying to add more than one option to mapreduce.map.java.opts without success. E.g.: yarn jar My.jar Myclass.class -Dmapreduce.map.java.opts=-Xss5M -agentlib:jdwp=transport=dt_socket,server=y,address=8787 -Dmapreduce.map.memory.mb=6000 ... followed by more options and maybe more than 2 arguments for mapreduce.map.java.opts. How do I have to do this that -Xss and -agentlib in my example are both treated as arguments for -Dmapreduce.map.memory.mb and not for the yarn jar job? I tried everything I could think of - single quotes, double quotes, ... Thanks for every hint, Eddie
... View more
Labels:
- Labels:
-
Apache Hadoop
09-27-2017
01:47 PM
I have the same problem (HDP 2.6, vmware). Reading a lot of related answers I believe that the problem is port forwarding and/or docker configuration. What has to be done to enable acess to a specific port (9999 in the example above) of the sandbox? (Probably the same is done for scp - the sandbox is accessed via port 2222 though there is nothing listening at port 2222 in the sandbox.) Unfortunately the most useful commands for examining this problem are not installed in the docker machine (e.g. ifconfig, netstat, ...)
... View more
07-12-2017
05:26 AM
The problem is virtualbox. With vmware everything is working as expected.
... View more
07-06-2017
06:22 AM
I tried to start the sandbox (HDP 2.6, 8 GB assigned) on a Core i7 with 16 GB RAM. Booting is not finished after more than 30 hours (CPU is very busy). Free RAM is available, according to the task manager VirtualBox is using about 124 MB of RAM only.
Is there anything I could do?
... View more