Support Questions
Find answers, ask questions, and share your expertise

Hive works fine for normal quering, but not for MAP-REDUCE job it fails. I have changed few property values too in the mapred-site i.e heap size, memory etc...

Hive works fine for normal quering, but not for MAP-REDUCE job it fails. I have changed few property values too in the mapred-site i.e heap size, memory etc...

Explorer
Error: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
        ... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
        ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
        ... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:140)
        ... 22 more
Caused by: java.lang.NoSuchFieldError: stringTypeInfo
        at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:115)
        at org.apache.hadoop.hive.serde2.SerDeUtils.initializeSerDe(SerDeUtils.java:527)
        at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:143)
        at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:304)
        at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:365)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:109)
        ... 22 more
3 REPLIES 3
Highlighted

Re: Hive works fine for normal quering, but not for MAP-REDUCE job it fails. I have changed few property values too in the mapred-site i.e heap size, memory etc...

What exactly have you changed? Does it work if you reset those properties to original values?

Highlighted

Re: Hive works fine for normal quering, but not for MAP-REDUCE job it fails. I have changed few property values too in the mapred-site i.e heap size, memory etc...

Could you please post your hive query? thanks

Highlighted

Re: Hive works fine for normal quering, but not for MAP-REDUCE job it fails. I have changed few property values too in the mapred-site i.e heap size, memory etc...

Rising Star

I think you are using SerDe's.

The hive might be looking for Serde jar to execute it in Hadoop.

Check your hive-default.xml for below property.

<property>
   <name>hive.aux.jars.path</name>
   <value>/your/serde/jar/path/in/hadoop</value>
</property>