Member since
06-05-2025
4
Posts
0
Kudos Received
0
Solutions
09-29-2025
09:22 AM
Reading back my post I didn't explain it very well. When I output a timestamp field using a FreeFormTextRecordSetWriter the field is implicitly converted to a string of format YYYY-mm-DD HH:MM:SS. I cannot see any way to change this behaviour and all my searches turn up blank. What I want is the Writer to respect the original long value.
... View more
09-26-2025
08:56 AM
Hitting a weird issue in Nifi that I can't get my head around. I am consuming AVRO records from Kafka with a timestamp field. The definition is {
"name" : "timestamp",
"type" : {
"type" : "long",
"logicalType" : "timestamp-millis"
}
}, If I use a JSON record writer then in the queue viewer I can see the field contains the original long value. However if I use an AVRO writer the field is formatted as a date/time value, e.g. 2025-09-26T15:29:44.466Z. I assumed this was just pretty-printing in the queue viewer. But if I then send those flowfiles to a ConvertRecord process with an AVRO reader and a FreeText writer the timestamp field is now displayed as YYYY-mm-DD HH:MM:SS. Which means I've lost precision and I need to use expression language to re-parse the field to convert it back to a long value. So, I'm forced to us JSON as the intermediary format. If I use AVRO I loose precision. This is Nifi 2.5.
... View more
Labels:
- Labels:
-
Apache NiFi
06-10-2025
02:33 AM
I'm publishing to Kafka 2.5 and running Nifi 2.4. There doesn't seem to be a choice in Nifi 2 for Kafka libraries - one size fits all? Anyway, not exactly sure what I did but suddenly I started getting errors reported to bulletin board and was able to diagnose and fix the underlying issue. It was a schema issue - missing fields without default value. I still don't get bulletins for successfully produced logs to Kafka, but from what you're saying that's just a limitation of the 3rd party libraries used?
... View more
06-05-2025
07:16 AM
I'm trying to construct a process group that produces to a Kafka topic. It's failing as all records are routed to the failure queue, but I get no errors from the PublishKafka processor to help me diagnose the problem. Digging into the PublishKafka process it seems that it doesn't produce any output whether on success or failure, despite having bulletin level set to DEBUG. If I substitute for a PutFile processor set to DEBUG then I get bulletins for each written file. So this looks like an issue specific to PublishKafka?
... View more
Labels:
- Labels:
-
Apache NiFi