Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎11-17-2015

Error running hive query which invokes mapreduce jobs

Hi

 

I am new to this community. I am working on Cloudera quickstart vm 5.4.2. I am facing error when running a hive query which invokes mapreduce jobs.

 

When I run

 

hive> select * from tweets_Cl;

 

it works fine but,

 

when I fire following query

 

hive> select count(*) from tweets_Cl;

 

I get error as below, I am not sure if it is something to do with configuration settings, please help! I did find similar issues on net but they couldn't be of much help to me.

 

Query ID = cloudera_20151117025151_15e9798e-ff9a-4a58-9d5a-66​db7454d70d
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1447755130552_0003, Tracking URL = http://quickstart.cloudera:8088/proxy/application_​1447755130552_0003/
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1447755130552_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2015-11-17 02:51:17,865 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:52:18,455 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:53:18,708 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:53:24,111 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_1447755130552_0003 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://quickstart.cloudera:8088/proxy/application_​1447755130552_0003/
Examining task ID: task_1447755130552_0003_m_000000 (and more) from job job_1447755130552_0003

Task with the most failures(4):
-----
Task ID:
  task_1447755130552_0003_m_000000

URL:
  http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1447​755130552_0003&tipid=task_1447755130552_0003_m_00...
-----
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:109)
        at org.apache.hadoop.util.ReflectionUtils.setConf(Ref​lectionUtils.java:75)
        at org.apache.hadoop.util.ReflectionUtils.newInstance​(ReflectionUtils.java:133)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapT​ask.java:449)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:​343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild​.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.do​As(UserGroupInformation.java:1671)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.​java:158)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ​e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Native​MethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(De​legatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:106)
        ... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:109)
        at org.apache.hadoop.util.ReflectionUtils.setConf(Ref​lectionUtils.java:75)
        at org.apache.hadoop.util.ReflectionUtils.newInstance​(ReflectionUtils.java:133)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRu​nner.java:38)
        ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ​e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Native​MethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(De​legatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:106)
        ... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.confi​gure(ExecMapper.java:147)
        ... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class com.cloudera.hive.serde.JSONSerDe not found
        at org.apache.hadoop.hive.ql.exec.MapOperator.getConv​ertedOI(MapOperator.java:323)
        at org.apache.hadoop.hive.ql.exec.MapOperator.setChil​dren(MapOperator.java:333)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.confi​gure(ExecMapper.java:116)
        ... 22 more
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.hive.serde.JSONSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByNam​e(Configuration.java:2018)
        at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDe​serializer(PartitionDesc.java:139)
        at org.apache.hadoop.hive.ql.exec.MapOperator.getConv​ertedOI(MapOperator.java:297)
        ... 24 more


FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

 

Many thanks in advance

 

Regards,

Sindhu

Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Error running hive query which invokes mapreduce jobs

Please use the inbuilt HCatalog JSONSerDe instead of a custom plugin, if it fits your data&colon; org.apache.hive.hcatalog.data.JsonSerDe. This will help avoid managing additional jars and classes.