Apache NiFi/ How to put data from kafka consumer to cassandra table ?

[#1]

I am using ConsumerKafka processor to pull data from putkafka. Now i want to put same data into cassandra table. But i am getting errors. I am using PutCassandraQL processor to put data into cassandra table.

Is there any way to put data directly from ConsumerKafka to PutCassandraQL.

Response will be appreciated. Thanks.

[#2]

There is currently no processor that can put the raw incoming data directly into Cassandra. For PutCassandraQL you would need to translate the data into CQL statements. This is often done by converting to JSON (if necessary), then using ConvertJSONToSQL (as of NiFi 1.5.0 via NIFI-4684 you can set the Prefix to "cql" vs the default "sql"), then PutCassandraQL.

I have written NIFI-5510 in Jira to cover the addition of a PutCassandraRecord processor, which should cover your use case of being able to go directly from ConsumeKafka to Cassandra.

[#3]

Hi @Matt Burgess thank you for your quick response.
I tried as per your comment but problem still remains.
Can you go through below attached image.
I am not getting data in ConvertJSONToSQL processor.
My main objective is to put data into cassandra table through NiFI flow.

[#4]

I forgot that ConvertJSONToSQL takes a DBCPConnectionPool as a required property, it is used to connect to the database to get table and column/type information in order to translate your JSON into SQL statements. You might be able to use the Cassandra JDBC driver and set up a DBCPConnectionPool using that driver.

An alternative is to use ConvertRecord with a JSONReader and a FreeFormTextRecordSetWriter, you might be able to generate the SQL statements yourself if you know the names and types of the fields. If you don't know the schema of the JSON, you can try InferAvroSchema first, but then I'm not sure what you'd be able to do with the FreeForm writer.

[#5]

Hi @Matt Burgess thanks for your quick response.

I tried as per your suggestion but the error remains same. Is there any template related to this.

[#6]

Thanks for this great community.

could you suggest any work flow to get data from GPS device to cassandra table with using Kafka.