Reply
Highlighted
Explorer
Posts: 8
Registered: ‎04-08-2014

cdh5.0.1 hive Job running in-process (local Hadoop) cause error? why

[ Edited ]

i test hive in cdh5.0.1, but i find hive always run in-process,not cluster, why?

 

because /etc/hive/conf is link to /etc/alternatives/hive-conf, 

/etc/alternatives/hive-conf is link to  /ect/hive/conf.cloudera.hive1

but /etc/hive/conf.cloudera.hive1 don't exists,  so i execute command:

cp -R /etc/hive/conf.cloudera.hive /etc/hive/conf.cloudera.hive1

 

hive> select count(*) from t_3a_cdr_list_export;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
14/05/27 13:09:19 WARN conf.Configuration: file:/tmp/root/hive_2014-05-27_13-09-15_315_2330811761484124655-1/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
14/05/27 13:09:19 WARN conf.Configuration: file:/tmp/root/hive_2014-05-27_13-09-15_315_2330811761484124655-1/-local-10002/jobconf.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
14/05/27 13:09:19 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
14/05/27 13:09:20 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
Execution log at: /tmp/root/root_20140527130909_e167dc47-3dce-48d2-8c95-bba333537d6d.log
Job running in-process (local Hadoop)
2014-05-27 13:09:24,900 null map = 0%, reduce = 0%
2014-05-27 13:09:27,949 null map = 100%, reduce = 0%
Ended Job = job_local1379109885_0001 with errors
Error during job, obtaining debugging information...
Execution failed with exit status: 2
Obtaining error information

Task failed!
Task ID:
Stage-1

Logs:

/tmp/root/hive.log
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

Posts: 1,903
Kudos: 435
Solutions: 305
Registered: ‎07-31-2013

Re: cdh5.0.1 hive Job running in-process (local Hadoop) cause error? why

Please add the gateway host (where you are running the hive command) as a MapReduce gateway role, and redeploy your cluster client configuration. This will help Hive in locating your actual remote cluster.

More on gateways at http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Managing-...