Member since
06-04-2020
7
Posts
0
Kudos Received
0
Solutions
06-16-2020
09:34 PM
I have seen in some article says timestamp only introduced in Avro 1.8.x. Anyone knows which avro version used in NIFI 1.11.4?
... View more
06-16-2020
05:18 AM
Hi, I was trying to fetch records from Cassandra's table with the help of Query Cassandra processor. I have timestamp field in the table for the date of birth and I store one record using the query INSERT INTO temp (id,dob,name)values (e7ae5cf3-d358-4d99-b900-85902fda9bb0, 1426349294842,'vinod'); While accessing this table in NIFI, I am getting the following error, need help here. [4:20 PM] E, Azhagesan
QueryCassandra[id=30619000-1586-3d6f-6d69-ed09ac1306d3] due to uncaught Exception: org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: Unknown datum type java.util.Date: Tue Jun 16 13:10:29 IST 2020
org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroRuntimeException: Unknown datum type java.util.Date: Tue Jun 16 13:10:29 IST 2020
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at org.apache.nifi.processors.cassandra.QueryCassandra.convertToAvroStream(QueryCassandra.java:358)
at org.apache.nifi.processors.cassandra.QueryCassandra$1.process(QueryCassandra.java:220)
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2746)
at org.apache.nifi.processors.cassandra.QueryCassandra.onTrigger(QueryCassandra.java:211)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.avro.AvroRuntimeException: Unknown datum type java.util.Date: Tue Jun 16 13:10:29 IST 2020
at org.apache.avro.generic.GenericData.getSchemaName(GenericData.java:741)
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:706)
at org.apache.avro.generic.GenericDatumWriter.resolveUnion(GenericDatumWriter.java:192)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:110)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 16 common frames omitted
... View more
Labels:
06-09-2020
01:55 AM
Great, thanks a lot
... View more
06-09-2020
12:44 AM
Nice solution this is what I was looking for, thanks. Query Cassandra designed to invoke in given scheduled time intervals right, Is there any way we can disable it because I need to fetch the value from Cassandra whenever the request comes from outside with the given employee id.
... View more
06-08-2020
10:14 PM
I was trying to fetch the data from a rest API inside NIFI, I have the following scenarios. 1) I have Rest API which fills fetch data from Cassandra database with the given parameters example employee number. 2) NIFI input port will receive this employee number from Flink application. How do I fetch data from the rest api with the given employee number received in the input port? I don't need a scheduling mechanism here, just need to fetch data from Rest API whenever a filter input received in the input port, is this possible using NIFI? I am trying this solution as Query cassandra will support only schedule based fetch, I have to fetch the data only when some inputs comes to Input port of NIFI template, please suggest if we have some alternative solutions..
... View more
Labels:
06-08-2020
10:11 PM
Hi Mahendra, thanks for the suggestions, this seems like I can not use Input port to receive in NIFI, Also can you help me with how can I transfer the parameter example "Emp No" in where clause of cassandra query in Query casandra?
... View more
06-05-2020
02:59 AM
I have a use case to read data from Cassandra where I provide inputs from outside NIFI[Flink Job], also instead of querying the given data for the scheduled time intervals, can we initiate the fetch whenever needed? Is these two use cases are possible with current NIFI system?
... View more
Labels: