- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
'Records Processed' missing for a processor
- Labels:
-
Apache NiFi
Created 09-14-2023 05:16 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
On a processor, right click -> view status history....there is a data available by name 'Records Processed' in dropdown.
For one of the processor in my flow, this is not available.
What can be the reason for this? Is it possible that value was so high that system lost track somewhere?
Created 09-15-2023 05:33 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@manishg
The Record Reader and record Writer controller services are not responsible for tracking counts. This is handled within your custom processors code. I am not sure what your custom processor does and whether it makes sense to track "record processed" or some other record based stat, but you can look at the github code for other processors like PartitionRecord to see how RecordCount is handled.
If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 09-14-2023 10:55 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@manishg
I am not clear, can you share the processor type where the expected data is not present?
The View Status History "Record Processed" will only appear on some component "Record" type processors (example: PartitionRecord processor). It would not exist as on processors that don't process "records".
If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 09-14-2023 07:28 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Its a custom processor, and it reads avro records(using AvroReader) and writes them too(using AvroRecordSetWriter).
Created 09-15-2023 05:33 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@manishg
The Record Reader and record Writer controller services are not responsible for tracking counts. This is handled within your custom processors code. I am not sure what your custom processor does and whether it makes sense to track "record processed" or some other record based stat, but you can look at the github code for other processors like PartitionRecord to see how RecordCount is handled.
If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
