Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Error running hive query which invokes mapreduce jobs

Error running hive query which invokes mapreduce jobs

New Contributor

Hi

 

I am new to this community. I am working on Cloudera quickstart vm 5.4.2. I am facing error when running a hive query which invokes mapreduce jobs.

 

When I run

 

hive> select * from tweets_Cl;

 

it works fine but,

 

when I fire following query

 

hive> select count(*) from tweets_Cl;

 

I get error as below, I am not sure if it is something to do with configuration settings, please help! I did find similar issues on net but they couldn't be of much help to me.

 

Query ID = cloudera_20151117025151_15e9798e-ff9a-4a58-9d5a-66​db7454d70d
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1447755130552_0003, Tracking URL = http://quickstart.cloudera:8088/proxy/application_​1447755130552_0003/
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_1447755130552_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2015-11-17 02:51:17,865 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:52:18,455 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:53:18,708 Stage-1 map = 0%,  reduce = 0%
2015-11-17 02:53:24,111 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_1447755130552_0003 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://quickstart.cloudera:8088/proxy/application_​1447755130552_0003/
Examining task ID: task_1447755130552_0003_m_000000 (and more) from job job_1447755130552_0003

Task with the most failures(4):
-----
Task ID:
  task_1447755130552_0003_m_000000

URL:
  http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1447​755130552_0003&tipid=task_1447755130552_0003_m_00...
-----
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:109)
        at org.apache.hadoop.util.ReflectionUtils.setConf(Ref​lectionUtils.java:75)
        at org.apache.hadoop.util.ReflectionUtils.newInstance​(ReflectionUtils.java:133)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapT​ask.java:449)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:​343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild​.java:163)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.do​As(UserGroupInformation.java:1671)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.​java:158)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ​e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Native​MethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(De​legatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:106)
        ... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:109)
        at org.apache.hadoop.util.ReflectionUtils.setConf(Ref​lectionUtils.java:75)
        at org.apache.hadoop.util.ReflectionUtils.newInstance​(ReflectionUtils.java:133)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRu​nner.java:38)
        ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ​e Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Native​MethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(De​legatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(​ReflectionUtils.java:106)
        ... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.confi​gure(ExecMapper.java:147)
        ... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class com.cloudera.hive.serde.JSONSerDe not found
        at org.apache.hadoop.hive.ql.exec.MapOperator.getConv​ertedOI(MapOperator.java:323)
        at org.apache.hadoop.hive.ql.exec.MapOperator.setChil​dren(MapOperator.java:333)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.confi​gure(ExecMapper.java:116)
        ... 22 more
Caused by: java.lang.ClassNotFoundException: Class com.cloudera.hive.serde.JSONSerDe not found
        at org.apache.hadoop.conf.Configuration.getClassByNam​e(Configuration.java:2018)
        at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDe​serializer(PartitionDesc.java:139)
        at org.apache.hadoop.hive.ql.exec.MapOperator.getConv​ertedOI(MapOperator.java:297)
        ... 24 more


FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

 

Many thanks in advance

 

Regards,

Sindhu

1 REPLY 1

Re: Error running hive query which invokes mapreduce jobs

Master Guru
Please use the inbuilt HCatalog JSONSerDe instead of a custom plugin, if it fits your data&colon; org.apache.hive.hcatalog.data.JsonSerDe. This will help avoid managing additional jars and classes.