Member since
08-13-2014
2
Posts
0
Kudos Received
0
Solutions
08-31-2014
06:27 PM
Thank you for your reply ! Detailed as follow: Log Type: stderr
Log Length: 242
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapred.TaskAttemptListenerImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Log Type: stdout
Log Length: 0
Log Type: syslog
Log Length: 339651
Showing 4096 bytes of 339651 total. Click here for the full log.
Size of the outstanding queue size is 0
2014-09-01 00:45:18,444 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://cloud60:8020/user/boco/.staging/job_1409203388885_0011/job_1409203388885_0011_1.jhist to hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011-1409503505152-boco-create+table+tab5_16+as%0Ase...initid_alarmid2%28Stage-1409503518377-3-0-FAILED-root.boco-1409503515788.jhist_tmp
2014-09-01 00:45:18,484 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011-1409503505152-boco-create+table+tab5_16+as%0Ase...initid_alarmid2%28Stage-1409503518377-3-0-FAILED-root.boco-1409503515788.jhist_tmp
2014-09-01 00:45:18,491 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://cloud60:8020/user/boco/.staging/job_1409203388885_0011/job_1409203388885_0011_1_conf.xml to hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011_conf.xml_tmp
2014-09-01 00:45:18,536 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied to done location: hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011_conf.xml_tmp
2014-09-01 00:45:18,549 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011.summary_tmp to hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011.summary
2014-09-01 00:45:18,555 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011_conf.xml_tmp to hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011_conf.xml
2014-09-01 00:45:18,559 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011-1409503505152-boco-create+table+tab5_16+as%0Ase...initid_alarmid2%28Stage-1409503518377-3-0-FAILED-root.boco-1409503515788.jhist_tmp to hdfs://cloud60:8020/user/history/done_intermediate/boco/job_1409203388885_0011-1409503505152-boco-create+table+tab5_16+as%0Ase...initid_alarmid2%28Stage-1409503518377-3-0-FAILED-root.boco-1409503515788.jhist
2014-09-01 00:45:18,561 INFO [Thread-75] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()
2014-09-01 00:45:18,562 ERROR [uber-EventHandler] org.apache.hadoop.mapred.LocalContainerLauncher: Returning, interrupted : java.lang.InterruptedException
2014-09-01 00:45:18,563 INFO [Thread-75] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Setting job diagnostics to Task failed task_1409203388885_0011_r_000000
Job failed as tasks failed. failedMaps:0 failedReduces:1
2014-09-01 00:45:18,563 INFO [Thread-75] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: History url is http://cloud60:19888/jobhistory/job/job_1409203388885_0011
2014-09-01 00:45:18,570 INFO [Thread-75] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Waiting for application to be successfully unregistered.
2014-09-01 00:45:19,572 INFO [Thread-75] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://cloud60:8020 /user/boco/.staging/job_1409203388885_0011
2014-09-01 00:45:19,581 INFO [Thread-75] org.apache.hadoop.ipc.Server: Stopping server on 58400
2014-09-01 00:45:19,582 INFO [IPC Server listener on 58400] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 58400
2014-09-01 00:45:19,583 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2014-09-01 00:45:19,586 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder
... View more
08-22-2014
01:10 AM
[boco@cloud92 ~]$ hive
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
14/08/22 15:56:39 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
14/08/22 15:56:40 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/hive/lib/hive-common-0.12.0-cdh5.1.0.jar!/hive-log4j.properties
hive> use datamine;
OK
Time taken: 0.687 seconds
hive> SELECT initid_alarmid1,
> COUNT (*) AS ab_num
> FROM tab5_11
> GROUP BY initid_alarmid1;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_1408689319506_0036, Tracking URL = http://cloud61:8088/proxy/application_1408689319506_0036/
Kill Command = /opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/lib/hadoop/bin/hadoop job -kill job_1408689319506_0036
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-08-22 15:57:10,729 Stage-1 map = 0%, reduce = 0%
2014-08-22 15:57:13,902 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 4.48 sec
2014-08-22 15:57:14,949 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 4.48 sec
MapReduce Total cumulative CPU time: 4 seconds 480 msec
Ended Job = job_1408689319506_0036 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1408689319506_0036_m_000000 (and more) from job job_1408689319506_0036
Task with the most failures(1):
-----
Task ID:
task_1408689319506_0036_r_000000
URL:
http://cloud61:8088/taskdetails.jsp?jobid=job_1408689319506_0036&tipid=task_1408689319506_0036_r_000000
-----
Diagnostic Messages for this Task:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:399)
at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:290)
at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:178)
at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:219)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 13 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:116)
... 18 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 Cumulative CPU: 4.48 sec HDFS Read: 111504306 HDFS Write: 179018 FAIL
Total MapReduce CPU Time Spent: 4 seconds 480 msec would anyone can help me? thank you
... View more
Labels: