Nifi Processor group level logging-Issue

@sarithe  NiFi component processors are part of pluggable nar in NiFi.  They are separate from the core NiFi code.  Processors are designed to log their output base on their component processor class.  It then becomes the responsibility of the logback to route those log messages to the appropriate appender.  There is nothing in the log output produced by a component processor that will inherently identify which parent process group it resides within.  But if you were to use a consistent processor naming structure in each of your Process Groups (PG), you may be able to setup some creative filtering in logback based on that naming structure.

Bulletins however do include details about the parent Process Group in which the component generating the bulletin resides.   You could build a dataflow in yoru NiFi to handle bulletin notification through the use of the SiteToSiteBulletinReportingTask which is used to send bulletin to a destination remote import port on a target NiFi.  A dataflow on the target NiFi could be built to parse the received bulletin records by the bulletinGroupName json path property so that all records from same PG are kept together.  These  'like' records could then be written out to local filesystem, remote system, used to send email notifications, etc...

Example of what a Bulletin sent using the SiteToSiteBulletinReportingTask looks like:

{
  "objectId" : "541dbd22-aa4b-4a1a-ad58-5d9a0b730e42",
  "platform" : "nifi",
  "bulletinId" : 2200,
  "bulletinCategory" : "Log Message",
  "bulletinGroupId" : "7e7ad459-0185-1000-ffff-ffff9e0b1503",
  "bulletinGroupName" : "PG2-Bulletin",
  "bulletinGroupPath" : "NiFi Flow / Matt's PG / PG2-Bulletin",
  "bulletinLevel" : "DEBUG",
  "bulletinMessage" : "UpdateAttribute[id=8c5b3806-9c3a-155b-ba15-260075ce9a6f] Updated attributes for StandardFlowFileRecord[uuid=1b0cb23a-75d8-4493-ba82-c6ea5c7d1ce3,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1672661850924-5, container=default, section=5], offset=969194, length=1024],offset=0,name=bulletin-${nextInt()).txt,size=1024]; transferring to 'success'",
  "bulletinNodeId" : "e75bf99f-095c-4672-be53-bb5510b3eb5c",
  "bulletinSourceId" : "8c5b3806-9c3a-155b-ba15-260075ce9a6f",
  "bulletinSourceName" : "PG1-UpdateAttribute",
  "bulletinSourceType" : "PROCESSOR",
  "bulletinTimestamp" : "2023-01-04T20:38:27.776Z"
}

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt