Support Questions

Find answers, ask questions, and share your expertise
Announcements
Welcome to the upgraded Community! Read this blog to see What’s New!

'Records Processed' missing for a processor

avatar
Contributor

On a processor, right click -> view status history....there is a data available by name 'Records Processed' in dropdown.

For one of the processor in my flow, this is not available.

What can be the reason for this? Is it possible that value was so high that system lost track somewhere?

1 ACCEPTED SOLUTION

avatar
Mentor

@manishg 
The Record Reader and record Writer controller services are not responsible for tracking counts.  This is handled within your custom processors code. I am not sure what your custom processor does and whether it makes sense to track "record processed" or some other record based stat, but you can look at the github code for other processors like PartitionRecord to see how RecordCount is handled.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

View solution in original post

3 REPLIES 3

avatar
Mentor

@manishg 
I am not clear, can you share the processor type where the expected data is not present?

The View Status History "Record Processed" will only appear on some component "Record" type processors (example: PartitionRecord processor).  It would not exist as on processors that don't process "records".  

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

avatar
Contributor

Its a custom processor, and it reads avro records(using AvroReader) and writes them too(using AvroRecordSetWriter).

avatar
Mentor

@manishg 
The Record Reader and record Writer controller services are not responsible for tracking counts.  This is handled within your custom processors code. I am not sure what your custom processor does and whether it makes sense to track "record processed" or some other record based stat, but you can look at the github code for other processors like PartitionRecord to see how RecordCount is handled.

If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Labels