wenjie
Contributor
My Accepted Solutions
Show All

Re: ERROR:is running beyond physical memory limits...

by Contributor in Support Questions
‎05-31-2017 06:26 PM
‎05-31-2017 06:26 PM
Hi mbigelow, Thanks for your reply. Appreciate it. ... View more

ERROR:is running beyond physical memory limits.

by Contributor in Support Questions
‎05-28-2017 07:50 PM
‎05-28-2017 07:50 PM
Hi ,    I got the error as below. It seems we need to set memory for  physical memory , but i can't find the configation about it. Where can i set physical memory?   But the way , i just run some simple insert query in HIVE,  and this error not happened on every time , i want to get the reason.   ERROR: Diagnostic Messages for this Task: Container [pid=51661,containerID=container_e50_1493005386967_25486_01_000243] is running beyond physical memory limits. Current usage: 1.0 GB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used. Killing container. Dump of the process-tree for container_e50_1493005386967_25486_01_000243 : |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE |- 51661 51659 51661 51661 (bash) 0 0 108662784 305 /bin/bash -c /usr/java/jdk1.8.0_101/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN.............     ... View more
Labels:

Re: How to limit speed ?

by Contributor in Support Questions
‎04-10-2017 07:05 PM
‎04-10-2017 07:05 PM
Sorry for late reply. I need to test it after business time.   I set it to 2046->1024->512-> 256 and found NetWork IO take a low value with the number reducing. I'm sure the parameter is working for it.   For someone who will refer this ticket. The speed of data transformation will take a very low speed for NetworkIO if you set a low value to this parameter. so please adjust schedul of jobs. ... View more

Re: How to limit speed ?

by Contributor in Support Questions
‎04-09-2017 06:35 PM
‎04-09-2017 06:35 PM
Yes, I set it from 4096 to 2048. The Network IO at the same level when it was 4096. Maybe i need to set less number than 2048? ... View more

Re: How to limit speed ?

by Contributor in Support Questions
‎04-06-2017 06:17 PM
‎04-06-2017 06:17 PM
It looks it( Maximum Number of Transfer Threads ) is not working.   I set it from 4096 to 2048 , but NetworkIO still take a high value. ... View more

Re: How to limit speed ?

by Contributor in Support Questions
‎04-03-2017 11:00 PM
‎04-03-2017 11:00 PM
Thanks Fawze. I will try it and take a feedback. ... View more

How to limit speed ?

by Contributor in Support Questions
‎04-03-2017 02:02 AM
‎04-03-2017 02:02 AM
Hi ,    I want to limit speed of data transformation with CDM because other process * (Not bigdata , not CDH) is using same internet resource with it(CDM). and the process* can't be run as no resource.    How can i set it in CHM? I tried to set 'DataNode Balancing Bandwidth' in HDFS configuration to 1Mb. But it looks not working.     ... View more
Labels:

Re: ERROR MESSAGE:This host is not in contact with...

by Contributor in Support Questions
‎03-22-2017 07:11 PM
‎03-22-2017 07:11 PM
I installed the host again , and the problem has been fixed.    I'm not sure the reason. ... View more

Re: ERROR MESSAGE:This host is not in contact with...

by Contributor in Support Questions
‎03-20-2017 10:58 PM
‎03-20-2017 10:58 PM
I created a new ticket for this problem as a summury, but i can't find it , why?   I move it to here.   I want to add a host to existing cluster as a datanode. But the host don't work fine as expected.  1, I confirmed status of agent and it is working fine. cloudera-scm-agent (pid  162575) is running... 2, I restarted the cluster and CDM , the problem can't be resolve. 3, I stopped agent and add the host and restart agent again ,  the problem can't be resolve also.   Looking forward to your response. And please let me know if you want more info.   Thanks, Wenjie   ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-20-2017 10:57 PM
‎03-20-2017 10:57 PM
NO , The problem has not be resolved. ... View more

ERROR MESSAGE:This host is not in contact with the...

by Contributor in Support Questions
‎03-20-2017 08:19 PM
‎03-20-2017 08:19 PM
Hi ,   I want to add a host to existing cluster as a datanode. But the host don't work fine as expected.      And i found the message in detail of host.   and the log    1, I confirmed status of agent and it is working fine. cloudera-scm-agent (pid  162575) is running... 2, I restarted the cluster and CDM , the problem can't be resolve. 3, I stopped agent and add the host and restart agent again ,  the problem can't be resolve also.   Looking forward to your response. And please let me know if you want more info.   Thanks, Wenjie     ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-17-2017 12:59 AM
‎03-17-2017 12:59 AM
I restarted the CDM and HOST MONITOR, it still can't be resloved. And i can't find CDH version in host list also for new host.   By the way , HOST MONITOR service and other datanode (existing) is working fine. so i guess some setting in new host is not good.   ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-17-2017 12:21 AM
‎03-17-2017 12:21 AM
I restarted the agent in host.   But can't reslove it .   And i found i can't get the CDH version ("None") in the host list. But existing datanode shows "CDH 5" ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-17-2017 12:11 AM
‎03-17-2017 12:11 AM
Hi csguna  ,   Working fine. I want to add a new datanode to existing system. and existing system is working fine . The added host not work well only. ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-17-2017 12:08 AM
‎03-17-2017 12:08 AM
I restarted host monitor , but it still not working. ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-16-2017 11:55 PM
‎03-16-2017 11:55 PM
And i confirmed the log of host and found it.   Caught unexpected exception in main loop. Traceback (most recent call last):   File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 710, in __issue_heartbeat     self._init_after_first_heartbeat_response(resp_data)   File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 881, in _init_after_first_heartbeat_response     assert response_data.get("cm_guid") is not None AssertionError       ... View more

Re: This host is not in contact with the Host Moni...

by Contributor in Support Questions
‎03-16-2017 11:47 PM
‎03-16-2017 11:47 PM
i'm sure i started agent in the host(datanode).   /etc/init.d/cloudera-scm-agent status cloudera-scm-agent (pid  112359) is running... ... View more

ERROR MESSAGE:This host is not in contact with the...

by Contributor in Support Questions
‎03-16-2017 11:39 PM
‎03-16-2017 11:39 PM
Hi ,   I want to add a host to cluster as a datanode.   Now i can find the host in host list and adding the New Hosts to existing Cluster. it looks it takes time because i began it 1hours ago but it is still running.   And i can find it in the cluster with new windows even the processing is running.   Bus Health test is bad. "This host is in contact with the Cloudera Manager Server. This host is not in contact with the Host Monitor."   How can i investigate it ?                   ... View more
Labels:

Re: How to add a data node to existing system in C...

by Contributor in Support Questions
‎03-13-2017 06:14 PM
‎03-13-2017 06:14 PM
Got it！ Thanks for your reply. ... View more

How to add a data node to existing system in Cloud...

by Contributor in Support Questions
‎03-13-2017 12:17 AM
‎03-13-2017 12:17 AM
Hi,    I want to add a data node to existing system ,  where can i find the manual or other?   I tried to find it in this site, but i couldn't find content related it.    Regards,  wenjie ... View more
Labels:

About HiveOptions.callTimeout

by Contributor in Support Questions
‎03-02-2017 07:57 PM
‎03-02-2017 07:57 PM
Hi,   I got error as below. ...... Launching Job 2 out of 10 Number of reduce tasks determined at compile time: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapreduce.job.reduces=<number> java.lang.RuntimeException: Error caching reduce.xml: java.nio.channels.ClosedByInterruptException at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:737) at org.apache.hadoop.hive.ql.exec.Utilities.setReduceWork(Utilities.java:676) at org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:667) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:374) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1782) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1539) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1318) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1127) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1115) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:172) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:383) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:318) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:720) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:693) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:628) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:478) at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:63) at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:159) at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:117) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:751) Job Submission failed with exception 'java.lang.RuntimeException(Error caching reduce.xml: java.nio.channels.Clos edByInterruptException)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched: ......   I investigate about it and found it is related with calltime option of HIVE. My question is ,  Where can i find this option in cloudera manager ? HiveOptions.callTimeout?   ... View more
Labels:

Re: IMPALA WEB UI NOT WORKING

by Contributor in Support Questions
‎02-26-2017 06:47 PM
‎02-26-2017 06:47 PM
@saranvisa Yes , you are right . I can't connect the host and i will apply it. Thanks a lot. ... View more

IMPALA WEB UI NOT WORKING

by Contributor in Support Questions
‎02-26-2017 06:05 PM
1 Kudo
‎02-26-2017 06:05 PM
1 Kudo
When i tried to open WEBUI of IMPARA in cloudera Manager , it's not working. It looks the page can't be found as i got a blank page with error message .   and i executed reflesh command with command mode in Linux also , it's working fine.    Could you please give me some idea about it? ... View more
Contact Me
Online
Offline
Last Visited
‎06-01-2017 01:56 AM
Public Statistics
Date Registered ‎02-26-2017 05:53 PM
Last Visited ‎06-01-2017 01:56 AM
Total Messages Posted 25
Total Kudos Received 1