Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Doing Math and replacingtext Line-byLine on ExtractText

Doing Math and replacingtext Line-byLine on ExtractText

Explorer

I have the below flowfile contents that I would like to manipulate without splitting the flowfile.  There may be a million of these lines.

{"ConfigBuild":"1007.8.0010703.9","ConfigStateHash":"1386539727","ContextProcessId":"317927403096194486","ContextThreadId":"0","ContextTimeStamp":"1603206060.314","Entitlements":"15","RawProcessId":"161622","TargetProcessId":"317927403096194486","aid":"1c41ff1d8eff49809ddb44d997ef2fa6","aip":"204.194.122.230","cid":"dff76f718745469994bd39c6a38579aa","event_platform":"Lin","event_simpleName":"TerminateProcess","id":"10c7f679-12e5-11eb-a7cc-0ab5dad5c022","name":"TerminateProcessLinV2","timestamp":"1603206060328"}
{"CommandLine":"logger -p cron.notice -t run-parts(/etc/cron.hourly)[115758] starting 0anacron","ConfigBuild":"1007.8.0010703.9","ConfigStateHash":"1386539727","Entitlements":"15","GID":"0","ImageFileName":"/usr/bin/logger","MD5HashData":"79922c9c30f225086d8a7331f54427ae","ParentBaseFileName":"bash","ParentProcessId":"317927531071092650","ProcessEndTime":"1603206061.046","ProcessGroupId":"317927531071092650","ProcessStartTime":"1603206061.045","RGID":"0","RUID":"0","RawProcessId":"115761","SHA1HashData":"0000000000000000000000000000000000000000","SHA256HashData":"9ed6a92880974f41eef1dd11b80d883c26faf1951a4b6a12720ccce8d195042c","SVGID":"0","SVUID":"0","SessionProcessId":"317927531071092652","SourceProcessId":"317927531071092650","SourceThreadId":"0","TargetProcessId":"317927531071092653","UID":"0","aid":"3a6f13022fff41a7b016537ea3862935","aip":"204.194.122.230","cid":"dff76f718745469994bd39c6a38579aa","event_platform":"Lin","event_simpleName":"ProcessRollup2","id":"11869eb7-12e5-11eb-9385-02ed8d7af942","name":"ProcessRollup2LinV6","timestamp":"1603206061577"}
{"CommandLine":"cat /var/spool/anacron/cron.daily","ConfigBuild":"1007.8.0010703.9","ConfigStateHash":"1386539727","Entitlements":"15","GID":"0","ImageFileName":"/usr/bin/cat","MD5HashData":"98b51c2a3efd088fe2ec632980c28e6c","ParentBaseFileName":"bash","ParentProcessId":"317927530540353195","ProcessEndTime":"1603206061.179","ProcessGroupId":"317927530540353191","ProcessStartTime":"1603206061.178","RGID":"0","RUID":"0","RawProcessId":"11630","SHA1HashData":"0000000000000000000000000000000000000000","SHA256HashData":"6825557ed76e1055d07a2cfd6ef753a931885daf4e102414724f9e812f48d538","SVGID":"0","SVUID":"0","SessionProcessId":"317927530540353193","SourceProcessId":"317927530540353195","SourceThreadId":"0","TargetProcessId":"317927532637505196","UID":"0","aid":"0a127135ef064baf8a4004991c43f2b7","aip":"204.194.122.230","cid":"dff76f718745469994bd39c6a38579aa","event_platform":"Lin","event_simpleName":"ProcessRollup2","id":"1199c591-12e5-11eb-b366-064134f5d9c0","name":"ProcessRollup2LinV6","timestamp":"1603206061703"}

In each row there is a timestamp, for example in the first row it is "timestamp":"1603206061703"

I'm doing an ExtractText manipulate the flowfile line by line with the goal of each line replaced with

 {"time": "THE_TIMESTAMP_VALUE/1000", "attribute a", "attribute_stuff_a", "event": $1}

So I want to do a calculation based on a value found in the line-by-line and add a couple attributes.  I've been able to do everything but add the calculated "time" value.

 

I tried separating the ExtractText and adding all the attributes in one and then adding the timestamp using the following ExtractText

2020-10-20_14-37-27.png

 
 
 
1 REPLY 1
Highlighted

Don't have to use ExtractText... maybe a groovyscript?

Explorer

Maybe this can't be done with ExtractText and there are other options

Don't have an account?
Coming from Hortonworks? Activate your account here