Member since
07-01-2021
1
Post
0
Kudos Received
0
Solutions
07-01-2021
06:20 PM
I have setup LogAttribute to log attribute and content to the nifi-app.log file. I would like to extract this data and log it to Kibana. I am able to get the normal logs to Kibana, but since the data coming from LogAttribute is multi-line the log is not being kept together and passed to elastic search. Example log: 2021-07-02 11:00:46,722 INFO [Timer-Driven Process Thread-3] o.a.n.processors.standard.LogAttribute LogAttribute[id=6025a30b-017a-1000-ee8d-77683321a3d8] logging for flow file StandardFlowFileRecord[uuid=c07c6741-498c-47e7-b2b1-2fbc9d8496ef,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1625186378831-37, container=default, section=37], offset=49133, length=914],offset=0,name=Sleepy.Medium.json,size=914] ---------CUSTOMLOG - Atom API - Response---------- Standard FlowFile Attributes Key: 'entryDate' Value: 'Fri Jul 02 11:00:46 AEST 2021' Key: 'lineageStartDate' Value: 'Thu Jul 01 13:15:20 AEST 2021' Key: 'fileSize' Value: '914' FlowFile Attribute Map Content Key: 'ComponentName' Value: 'Sleepy.Medium' Key: 'ComponentVersion' Value: '1.0.45' Key: 'DeploymentId' Value: 'Deployments-230644' Key: 'DeploymentSource' Value: 'http://ddd/api/Spaces-1' Key: 'EventTime' Value: '2021-07-01 03:15:16.817' Key: 'MachineName' Value: 'DER1533' Key: 'ProjectName' Value: 'Trigger Test' Key: 'ReleaseVersion' Value: '0.0.6' Key: 'invokehttp.request.url' Value: 'http://ddd:32717/api/Graph' Key: 'invokehttp.status.code' Value: '200' ---------CUSTOMLOG - Atom API - Response---------- {"appliedGraphChanges":{"nodes":[{"componentId":0,"name":"Sleepy.Medium","version":null,"type":null,"repositoryName":null,"score":null,"isDeprecated":null,"isRegMarketCritical":null,"isShared":null,"isDatabase":null,"isProcess":null,"isClient":null,"dateAdded":null,"lastUpdated":null,"dateDeleted":null,"isDeleted":false,"databaseType":null,"dependencies":null,"nodeType":2},{"name":"DERxxxx","memorySize":0.0,"numberCPUs":0,"provisionedDiskSpace":0.0,"ipAddress":null,"dateAdded":null,"lastUpdated":null,"dateDeleted":null,"isDeleted":false,"nodeType":7}],"edges":[{"from":{"nodeType":2,"nodeName":"Sleepy.Medium"},"to":{"nodeType":7,"nodeName":"DERxxxx"},"componentVersion":"1.0.45","deploymentDate":"2021-07-01T03:15:16.817","dateAdded":null,"lastUpdated":null,"dateDeleted":null,"isDeleted":false}],"metaData":{},"rawPayload":null},"deniedGraphChanges":{"nodes":[],"edges":[],"metaData":{},"rawPayload":null}}
... View more
Labels:
- Labels:
-
Apache NiFi