Hi, we dont have yet, but the smartsense is free?? Thanks ... View more

Hi: after the execution,i saw this graphic but i think its normal??? i dont know why at this moment after minutes get the error 16/02/17 10:02:52 INFO mapreduce.Job: map 99% reduce 33%16/02/17 10:02:56 INFO mapreduce.Job: map 100% reduce 33%16/02/17 10:03:01 INFO mapreduce.Job: map 100% reduce 67% ... View more

Also how can i set this parameter?? 16/02/17 09:57:09 INFO mapreduce.JobSubmitter: number of splits:107 Thanks alot ... View more

Also how can i set this parameter?? 16/02/17 09:57:09 INFO mapreduce.JobSubmitter: number of splits:107 Thanks alot ... View more

Also this error: 16/02/17 00:09:02 INFO mapreduce.Job: map 100% reduce 0% 16/02/17 00:09:13 INFO mapreduce.Job: map 100% reduce 67% 16/02/17 00:18:53 INFO mapreduce.Job: Task Id : attempt_1455662313758_0003_r_000000_2, Status : FAILED Container [pid=24748,containerID=container_e34_1455662313758_0003_01_000111] is running beyond physical memory limits. Current usage: 3.5 GB of 3.5 GB physical memory used; 10.7 GB of 14 GB virtual memory used. Killing container. Dump of the process-tree for container_e34_1455662313758_0003_01_000111 : |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 24748 24745 24748 24748 (bash) 0 0 108617728 341 /bin/bash -c /usr/jdk64/jdk1.8.0_40/bin/java -server -XX:NewRatio=8 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.3.2.0-2950 -Xmx5734m -Djava.io.tmpdir=/hadoop/yarn/local/usercache/dangulo/appcache/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/yarn/log/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 10.1.246.20 52488 attempt_1455662313758_0003_r_000000_2 37383395344495 1>/hadoop/yarn/log/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111/stdout 2>/hadoop/yarn/log/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111/stderr |- 24847 24835 24748 24748 (cat) 0 22 103407616 173 cat |- 24758 24748 24748 24748 (java) 1124 305 8068534272 227250 /usr/jdk64/jdk1.8.0_40/bin/java -server -XX:NewRatio=8 -Djava.net.preferIPv4Stack=true -Dhdp.version=2.3.2.0-2950 -Xmx5734m -Djava.io.tmpdir=/hadoop/yarn/local/usercache/dangulo/appcache/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/hadoop/yarn/log/application_1455662313758_0003/container_e34_1455662313758_0003_01_000111 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 10.1.246.20 52488 attempt_1455662313758_0003_r_000000_2 37383395344495 |- 24835 24758 24748 24748 (R) 57497 677 3131715584 700751 /usr/lib64/R/bin/exec/R --slave --no-restore --vanilla --file=./rmr-streaming-reduce5178597eff5e |- 24851 24835 24748 24748 (cat) 0 0 103407616 165 cat ... View more

Hi: After that, still doenst work, all the mapper finished correctly but in the reducer stop in 67% packageJobJar: [] [/usr/hdp/2.3.2.0-2950/hadoop-mapreduce/hadoop-streaming-2.7.1.2.3.2.0-2950.jar] /tmp/streamjob2854666121307172018.jar tmpDir=null 16/02/17 09:57:07 INFO impl.TimelineClientImpl: Timeline service address: http://lnxbig06.cajarural.gcr:8188/ws/v1/timeline/ 16/02/17 09:57:07 INFO client.RMProxy: Connecting to ResourceManager at lnxbig05.cajarural.gcr/10.1.246.19:8050 16/02/17 09:57:08 INFO impl.TimelineClientImpl: Timeline service address: http://lnxbig06.cajarural.gcr:8188/ws/v1/timeline/ 16/02/17 09:57:08 INFO client.RMProxy: Connecting to ResourceManager at lnxbig05.cajarural.gcr/10.1.246.19:8050 16/02/17 09:57:08 INFO mapred.FileInputFormat: Total input paths to process : 14 16/02/17 09:57:08 INFO net.NetworkTopology: Adding a new node: /default-rack/10.1.246.17:50010 16/02/17 09:57:08 INFO net.NetworkTopology: Adding a new node: /default-rack/10.1.246.18:50010 16/02/17 09:57:08 INFO net.NetworkTopology: Adding a new node: /default-rack/10.1.246.16:50010 16/02/17 09:57:08 INFO net.NetworkTopology: Adding a new node: /default-rack/10.1.246.20:50010 16/02/17 09:57:09 INFO mapreduce.JobSubmitter: number of splits:107 16/02/17 09:57:09 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1455692856660_0019 16/02/17 09:57:09 INFO impl.YarnClientImpl: Submitted application application_1455692856660_0019 16/02/17 09:57:09 INFO mapreduce.Job: The url to track the job: http://lnxbig05.cajarural.gcr:8088/proxy/application_1455692856660_0019/ 16/02/17 09:57:09 INFO mapreduce.Job: Running job: job_1455692856660_0019 16/02/17 09:57:15 INFO mapreduce.Job: Job job_1455692856660_0019 running in uber mode : false 16/02/17 09:57:15 INFO mapreduce.Job: map 0% reduce 0% 16/02/17 09:57:39 INFO mapreduce.Job: map 1% reduce 0% 16/02/17 09:57:40 INFO mapreduce.Job: map 2% reduce 0% 16/02/17 09:57:41 INFO mapreduce.Job: map 3% reduce 0% 16/02/17 09:57:42 INFO mapreduce.Job: map 5% reduce 0% 16/02/17 09:57:43 INFO mapreduce.Job: map 6% reduce 0% 16/02/17 09:57:44 INFO mapreduce.Job: map 8% reduce 0% 16/02/17 09:57:45 INFO mapreduce.Job: map 13% reduce 0% 16/02/17 09:57:46 INFO mapreduce.Job: map 15% reduce 0% 16/02/17 09:57:47 INFO mapreduce.Job: map 17% reduce 0% 16/02/17 09:57:48 INFO mapreduce.Job: map 20% reduce 0% 16/02/17 09:57:49 INFO mapreduce.Job: map 22% reduce 0% 16/02/17 09:57:50 INFO mapreduce.Job: map 24% reduce 0% 16/02/17 09:57:51 INFO mapreduce.Job: map 27% reduce 0% 16/02/17 09:57:52 INFO mapreduce.Job: map 29% reduce 0% 16/02/17 09:57:53 INFO mapreduce.Job: map 31% reduce 0% 16/02/17 09:57:54 INFO mapreduce.Job: map 34% reduce 0% 16/02/17 09:57:55 INFO mapreduce.Job: map 35% reduce 0% 16/02/17 09:57:56 INFO mapreduce.Job: map 37% reduce 0% 16/02/17 09:57:57 INFO mapreduce.Job: map 41% reduce 0% 16/02/17 09:57:58 INFO mapreduce.Job: map 44% reduce 0% 16/02/17 09:57:59 INFO mapreduce.Job: map 45% reduce 0% 16/02/17 09:58:00 INFO mapreduce.Job: map 48% reduce 0% 16/02/17 09:58:01 INFO mapreduce.Job: map 51% reduce 0% 16/02/17 09:58:03 INFO mapreduce.Job: map 54% reduce 0% 16/02/17 09:58:04 INFO mapreduce.Job: map 56% reduce 0% 16/02/17 09:58:06 INFO mapreduce.Job: map 57% reduce 0% 16/02/17 09:58:07 INFO mapreduce.Job: map 58% reduce 0% 16/02/17 10:01:16 INFO mapreduce.Job: map 59% reduce 0% 16/02/17 10:01:21 INFO mapreduce.Job: map 60% reduce 0% 16/02/17 10:01:24 INFO mapreduce.Job: map 61% reduce 0% 16/02/17 10:01:28 INFO mapreduce.Job: map 62% reduce 0% 16/02/17 10:01:31 INFO mapreduce.Job: map 63% reduce 0% 16/02/17 10:01:32 INFO mapreduce.Job: map 64% reduce 0% 16/02/17 10:01:33 INFO mapreduce.Job: map 65% reduce 0% 16/02/17 10:01:35 INFO mapreduce.Job: map 66% reduce 0% 16/02/17 10:01:37 INFO mapreduce.Job: map 67% reduce 0% 16/02/17 10:01:38 INFO mapreduce.Job: map 68% reduce 0% 16/02/17 10:01:40 INFO mapreduce.Job: map 69% reduce 0% 16/02/17 10:01:42 INFO mapreduce.Job: map 70% reduce 8% 16/02/17 10:01:43 INFO mapreduce.Job: map 71% reduce 8% 16/02/17 10:01:44 INFO mapreduce.Job: map 73% reduce 8% 16/02/17 10:01:46 INFO mapreduce.Job: map 73% reduce 9% 16/02/17 10:01:47 INFO mapreduce.Job: map 74% reduce 9% 16/02/17 10:01:48 INFO mapreduce.Job: map 75% reduce 9% 16/02/17 10:01:49 INFO mapreduce.Job: map 76% reduce 10% 16/02/17 10:01:50 INFO mapreduce.Job: map 77% reduce 10% 16/02/17 10:01:51 INFO mapreduce.Job: map 78% reduce 10% 16/02/17 10:01:52 INFO mapreduce.Job: map 80% reduce 12% 16/02/17 10:01:53 INFO mapreduce.Job: map 82% reduce 12% 16/02/17 10:01:55 INFO mapreduce.Job: map 84% reduce 15% 16/02/17 10:01:56 INFO mapreduce.Job: map 86% reduce 15% 16/02/17 10:01:57 INFO mapreduce.Job: map 88% reduce 15% 16/02/17 10:01:58 INFO mapreduce.Job: map 90% reduce 21% 16/02/17 10:01:59 INFO mapreduce.Job: map 92% reduce 21% 16/02/17 10:02:00 INFO mapreduce.Job: map 93% reduce 21% 16/02/17 10:02:01 INFO mapreduce.Job: map 93% reduce 25% 16/02/17 10:02:02 INFO mapreduce.Job: map 94% reduce 25% 16/02/17 10:02:03 INFO mapreduce.Job: map 96% reduce 25% 16/02/17 10:02:04 INFO mapreduce.Job: map 96% reduce 29% 16/02/17 10:02:06 INFO mapreduce.Job: map 97% reduce 29% 16/02/17 10:02:07 INFO mapreduce.Job: map 97% reduce 30% 16/02/17 10:02:19 INFO mapreduce.Job: map 97% reduce 31% 16/02/17 10:02:25 INFO mapreduce.Job: map 98% reduce 31% 16/02/17 10:02:37 INFO mapreduce.Job: map 98% reduce 32% 16/02/17 10:02:38 INFO mapreduce.Job: map 99% reduce 32% 16/02/17 10:02:52 INFO mapreduce.Job: map 99% reduce 33% 16/02/17 10:02:56 INFO mapreduce.Job: map 100% reduce 33% 16/02/17 10:03:01 INFO mapreduce.Job: map 100% reduce 67% ... View more

Hi: Still in 500 mb doesnt work, but i dont understand why I need to down this memory, please could you explain me in detail?? Many thanks ... View more

ok let me try and ill tell which parameter i used 🙂 Many thanks again. ... View more

Hi: i cant put more than 2047 mb, and stil doesnt work..... ... View more