Reply
New Contributor
Posts: 2
Registered: ‎08-27-2015

Hi,Hadoop and spring configuration

Hi,

 

I am trying to run hadoop using spring 4.2 in eclips.  I have configure and write simple wordcount programe. but when i am running it. I am getting excption

I had googled for soluton but conldn't fild the solution. Could some one help me out..

 

 

Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'wordCountJobRunner': Invocation of init method failed; nested exception is java.lang.ExceptionInInitializerError
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1578)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:545)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:305)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:301)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:196)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:772)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:834)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:537)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:83)
    at com.hadoop.max.temp.DemoMain.main(DemoMain.java:8)
Caused by: java.lang.ExceptionInInitializerError
    at org.springframework.data.hadoop.mapreduce.JobExecutor$2.run(JobExecutor.java:183)
    at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50)
    at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:168)
    at org.springframework.data.hadoop.mapreduce.JobExecutor.startJobs(JobExecutor.java:160)
    at org.springframework.data.hadoop.mapreduce.JobRunner.call(JobRunner.java:52)
    at org.springframework.data.hadoop.mapreduce.JobRunner.afterPropertiesSet(JobRunner.java:44)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1637)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1574)
    ... 12 more
Caused by: java.lang.IllegalStateException: Invalid Job.class detected, probably caused by a Hadoop YARN library, which is NOT supported yet.
See the Requirements chapter in the reference documentation for more information.
    at org.springframework.data.hadoop.mapreduce.JobUtils.<clinit>(JobUtils.java:124)
    ... 20 more

 

Thanks

Posts: 1,836
Kudos: 416
Solutions: 295
Registered: ‎07-31-2013

Re: Hi,Hadoop and spring configuration

What Hadoop libraries are you running with? Spring looks for a "state" field/"getJobState" method under the classpath-loaded org.apache.hadoop.mapreduce.Job class, but is unable to find it in your case.

If you are copying jars out of the VM, ensure you use /usr/lib/hadoop-mapreduce/ (MR2, newer, what Spring wants) and not /usr/lib/hadoop-0.20-mapreduce/ (MR1, older, provided for compatibility, not what Spring wants)