I am trying to build a Kafka Producer using Java and Maven.

[#1]

I am trying to build a Kafka Producer using Java and Maven. According to this article (https://community.hortonworks.com/articles/51122/simple-kafka-producer-using-java-in-a-kerberozied.html) I have to modify the pom.xml file in order to use the org.apache.kafka dependency. Unnfortunatelly, when I build the project, the following error is shown:

ERROR:

[INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building abud 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ abud --- [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /root/Documents/MavenExamples/abud/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:2.0.2:compile (default-compile) @ abud --- [INFO] Compiling 1 source file to /root/Documents/MavenExamples/abud/target/classes [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 0.607s [INFO] Finished at: Sun Jan 29 00:49:27 EST 2017 [INFO] Final Memory: 15M/483M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:compile (default-compile) on project abud: Compilation failure [ERROR] /root/Documents/MavenExamples/abud/src/main/java/com/tecnisys/App.java:[20,19] error: generics are not supported in -source 1.3 [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

,

[#2]

@Rafael Menezes

The reason for build failure seems to be the following:

[ERROR] /root/Documents/MavenExamples/abud/src/main/java/com/tecnisys/App.java:[20,19] error: generics are not supported in -source 1.3 [ERROR] -> [Help 1] 

.

Which indicates that in your "com.tecnisys.App" (App.java) code you are using Java Generics feature which is not available in "Source 1.3" . Hence please check your pom.xml file to see if you are using any "maven-compiler-plugin" there. If yes then try changing the "source" and "target" version to 1.6/1.7 ..etc

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <version>2.3</version>
  <configuration>
    <source>1.6</source>
    <target>1.6</target>
  </configuration>
</plugin>

.

[#3]

Thanks, Jay. Now it works, but instead of 1.6 I put 1.8.

Unfortunatelly now I have a new error. The compiler says:

[ERROR] /root/Documents/MavenExamples/abud/src/main/java/com/tecnisys/App.java:[25,2] error: cannot find symbol

The code is:

package com.tecnisys; import java.util.*;

public class App { public static void main( String[] args ) {

System.out.println( "Hello Life!" );

Properties properties = new Properties();

properties.put("metadata.broker.list", "localhost:9092");

properties.put("bootstrap.servers","comma-seperated-list-of-brokers"); properties.put("key.serializer","org.apache.kafka.common.serialization.StringSerializer");// key serializer properties.put("value.serializer","org.apache.kafka.common.serialization.StringSerializer"); //value serializer properties.put("acks","1"); //message durability -- 1 mean ack after writing to leader is success. value of "all" means ack after replication.

properties.put("security.protocol","SASL_PLAINTEXT"); // Security protocol to use for communication. properties.put("batch.size","16384");// maximum size of message

KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties); } }

[#4]

@Rafael Menezes

Good to know that initial issue is resolved. You can use 1.6/ 1.7/ 1.8 anything there (but not any version prior to 1.5). Because Generics feature was introduce in prior to 1.5 version.

Regarding your new compilation error , we will need to exactly see the line number "25" of your code "App.java"

[ERROR] /root/Documents/MavenExamples/abud/src/main/java/com/tecnisys/App.java:[25,2] error: cannot find symbol

.

Can you please paste the code in correct formatting so that we can see what is there at line "25" that is causing compilation error.

.

[#5]

Jay,

thanks for the very quick reply. Here is the code I put in line 25:

KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties);

[#6]

@Rafael Menezes

Also looks like in your code the following import statement is missing:

import org.apache.kafka.clients.producer.KafkaProducer;

.

Please add proper dependency for this class in your pom.xml as well. Something like following:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>${YOUR_DESIRED_VERSION}</version>
</dependency>

.

[#7]

The POM was Ok.

I only put import org.apache.kafka.clients.producer.KafkaProducer;

I thought that the mention in the pom would be enough. Until this point it is ok. Thanks a lot, Jay!

[#8]

Dear Jay,

I couldnt use the KafkaProducer class. The following erros was shown:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/KafkaProducer at com.tecnisys.App.main(App.java:27) Caused by: java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.KafkaProducer at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more

I import org.apache.kafka.clients.producer.KafkaProducer and put org.apache.kafka as a dependency in pom.xml.

[#9]

@Rafael Menezes

Good to see that now you are able to compile your code/build fine.

However this time the error is more of "RunTime" (not compile time). As following:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/KafkaProducer 
at com.tecnisys.App.main(App.java:27) 
Caused by: 
java.lang.ClassNotFoundException: org.apache.kafka.clients.producer.KafkaProducer 

Your classpath is not correctly pointing to the kafka-client jar.

Please see a complete demo here:

https://github.com/mapr-demos/kafka-sample-programs/blob/master/pom.xml

.

If you are running your code manually then you might do the following:

export CLASSPATH = $CLASSPATH:/PATH/TO/kafka-client.jar:.:
java com.tecnisys.App

.

[#10]

Jay,

maybe I need to study a lot more Maven, but in at my knowledge this POM is enough to reference kafka-clients, isn't it?

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion>

<groupId>com.tecnisys</groupId>

<artifactId>abud</artifactId>

<packaging>jar</packaging>

<version>1.0-SNAPSHOT</version>

<name>abud</name>

<url>http://maven.apache.org</url>

<dependencies>

<dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

<version>3.8.1</version>

<scope>test</scope>

</dependency>

<dependency>

<groupId>org.apache.kafka</groupId>

<artifactId>kafka-clients</artifactId>

<version>0.10.1.1</version>

</dependency>

</dependencies>

<build>

<plugins>

<plugin>

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-compiler-plugin</artifactId>

<version>2.3.2</version>

<configuration>

<source>1.8</source>

<target>1.8</target>

</configuration>

</plugin>

</plugins>

</build>

</project>