Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Jar is not getting loaded from user classpath in hadoop job

Jar is not getting loaded from user classpath in hadoop job

New Contributor

Hello,

While launching MR job below error(given at the end) is occurring. By seeing error, looks like its httpClient jar version issue.

My jar has "httpclient-4.5.1.jar" under lib directory. Used below parameter so that this jar gets picked from my jar file.

-Dmapreduce.user.claspath.first=true

After enabling verbose, I can see jar is getting loaded from hadoop dir (/opt/cloudera/parcels/CDH-5.4.5-1.cdh5.4.5.p0.7/jars/httpclient-4.2.5.jar).

HADOOP_CLIENT_OPTS="-verbose:class"

Tried other options as well like -libjars, HADOOP_CLASSPATH, export HADOOP_USER_CLASSPATH_FIRST=true etc.

But they do not seem to be working.

Error:

-----------------------------

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.audiencescience.apollo.util.client.http.BasicAuthClientHttpRequestFactory#d9790d9' defined in class path resource [apolloGsonAndRestTemplate.xml]: Cannot create inner bean 'com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory#68d93a9e' of type [com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory#68d93a9e' defined in class path resource [apolloGsonAndRestTemplate.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory]: Constructor threw exception; nested exception is java.lang.NoSuchFieldError: INSTANCE

at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:313)

at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)

at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:648)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1143)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)

at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)

... 40 more

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory#68d93a9e' defined in class path resource [apolloGsonAndRestTemplate.xml]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory]: Constructor threw exception; nested exception is java.lang.NoSuchFieldError: INSTANCE

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:275)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1143)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1046)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:510)

at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:482)

at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:299)

... 48 more

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory]: Constructor threw exception; nested exception is java.lang.NoSuchFieldError: INSTANCE

at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:163)

at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:122)

at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:267)

... 53 more

Caused by: java.lang.NoSuchFieldError: INSTANCE

at org.apache.http.conn.ssl.SSLConnectionSocketFactory.<clinit>(SSLConnectionSocketFactory.java:144)

at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:955)

at com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory.createHttpClient(ConnectionPoolingClientHttpRequestFactory.java:26)

at com.audiencescience.apollo.util.client.http.ConnectionPoolingClientHttpRequestFactory.<init>(ConnectionPoolingClientHttpRequestFactory.java:15)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:526)

at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)

... 55 more

Thanks

Shubham