Support Questions
Find answers, ask questions, and share your expertise

NoClassDefFoundError for Log4jLoggerFactory on hdp 2.5.3 when running the KafkaSpout in my storm topology

New Contributor

I am trying to deploy a my storm topology on cluster. I am using kafka as the input(kafka Spout).This is the error i am getting

java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory at org.apache.log4j.Logger.getLogger(Logger.java:39) at kafka.utils.Logging$class.logger(Logging.scala:24) at kafka.consumer.SimpleConsumer.logger$lzycompute(SimpleConsumer.scala:35) at kafka.consumer.SimpleConsumer.logger(SimpleConsumer.scala:35) at kafka.utils.Logging$class.info(Logging.scala:75) at kafka.consumer.SimpleConsumer.info(SimpleConsumer.scala:35) at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:94) at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:83) at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:149) at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79) at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:75) at org.apache.storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:65) at org.apache.storm.kafka.PartitionManager.<init>(PartitionManager.java:94) at org.apache.storm.kafka.ZkCoordinator.refresh(ZkCoordinator.java:98) at org.apache.storm.kafka.ZkCoordinator.getMyManagedPartitions(ZkCoordinator.java:69) at org.apache.storm.kafka.KafkaSpout.nextTuple(KafkaSpout.java:129) at org.apache.storm.daemon.executor$fn__6505$fn__6520$fn__6551.invoke(executor.clj:651) at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745)

My pom.xml looks like this:-

 <dependencies>
   <dependency>
     <groupId>org.apache.storm</groupId>
      <artifactId>storm-hdfs</artifactId>
       <version>1.0.1</version>
	<exclusions>
	   <exclusion>
		<groupId>org.slf4j</groupId>
		<artifactId>slf4j-log4j12</artifactId>
	   </exclusion>
	   <exclusion>
		<groupId>log4j</groupId>
		<artifactId>log4j</artifactId>
	   </exclusion>
	</exclusions>
   </dependency>
   <dependency>
       <groupId>org.apache.kafka</groupId>
       <artifactId>kafka_2.10</artifactId>
       <version>0.9.0.1</version>
    <exclusions>
      <exclusion>
	<groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
      </exclusion>
      <exclusion>
	<groupId>log4j</groupId>
	<artifactId>log4j</artifactId>
      </exclusion>
    </exclusions>
  </dependency>
  <dependency>
     <groupId>org.apache.kafka</groupId>
     <artifactId>kafka-clients</artifactId>
     <version>0.9.0.1</version>
     <exclusions>
	<exclusion>
		<groupId>org.slf4j</groupId>
		<artifactId>slf4j-log4j12</artifactId>
	</exclusion>
	<exclusion>
		<groupId>log4j</groupId>
		<artifactId>log4j</artifactId>
	</exclusion>
      </exclusions>
   </dependency>
   <dependency>
	<groupId>org.apache.storm</groupId>
	<artifactId>storm-kafka</artifactId>
 	<version>1.0.1</version>
   </dependency>
   <dependency>
            <groupId>org.apache.storm</groupId>
            <artifactId>storm-core</artifactId>
            <scope>provided</scope>
            <version>1.0.1</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-auth -->
		<dependency>
		    <groupId>org.apache.hadoop</groupId>
		    <artifactId>hadoop-auth</artifactId>
		    <version>2.2.0</version>
		</dependency>
        <dependency>
            <groupId>org.apache.storm</groupId>
            <artifactId>storm-redis</artifactId>
            <version>1.0.1</version>
		<exclusions>
			<exclusion>
				<groupId>org.slf4j</groupId>
				<artifactId>slf4j-log4j12</artifactId>
			</exclusion>
			<exclusion>
				<groupId>log4j</groupId>
				<artifactId>log4j</artifactId>
			</exclusion>
		</exclusions>
        </dependency>
    </dependencies>
1 REPLY 1

Super Mentor

@Aditi Kumari

You are getting "NoClassDefFoundError" .. error.

java.lang.NoClassDefFoundError: Could not initialize class org.apache.log4j.Log4jLoggerFactory

.

Is there any specific reason you are <excluding> the following log4j artifacts?

<exclusion>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
</exclusion>

.

Can you remove that exclusion block and try again.

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.