Member since
07-25-2018
5
Posts
0
Kudos Received
0
Solutions
07-12-2019
03:23 PM
I have a txt file (attached) and I would like split the lines and to parse out the data and set the values to flowfile attributes (PPGRAW : 1.76, PPGCORR : 5.15, etc.). Ideally I would like to just get the very last row of values (ignoring the last couple rows of "878 LOG,STOP" and ignoring that first numeric value separated by the space) and set those values to flowfile attributes accordingly. I was able to split the original text file by line to include headers (first image). I can split them to not include the headers (since I already know which values are for which properties - second image). My flow consists of GetFile -> SplitText for now. m6_sample.txt
... View more
Labels:
06-17-2019
03:21 PM
DetectDuplicate error "unable to communicate with cache when processing due to connection refused." Retrieving data from mongo collection, setting flowfile attribute "oid" to the mongo generated id given to each entry from the collection, then detecting any duplicate data based on that oid for when GetMongo runs again.
... View more
Labels:
03-13-2019
09:48 PM
@Matt Burgess I just wanted to clarify your 2 recipes for getting incoming flowfiles. The first mentions getting 1 incoming flow file and the second mentions getting multiple. If there are multiple flow files in queue, but you want the script to modify all flow files but only one at a time, which recipe is the one to follow? I'm assuming the first recipe is appropriate for this but wanted to clarify. Thanks
... View more
03-13-2019
01:24 PM
multiple_json_not_array.txt
... View more
07-25-2018
03:12 PM
Hi Matt, I am trying to use the RunMongoAggregation processor and ran into 2 issues. 1. I ran an aggregation in my mongo very similar to this:
[ { $project: { lastUpdated: { $dateFromString: { dateString: '$lastUpdated' } } } } ] It changes the "lastUpdated" value from a string type to a DateTime type. The result looks like this:
lastUpdated 2018-07-23T10:15:05Z DateTime
I tried this in the RunMongoAggregation and with double quotes around each field as this link suggests https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.6.0/org.apache.nifi.processors.mongodb.RunMongoAggregation/additionalDetails.html But the result I got looks like this:
lastUpdated NumberLong("1532340905000") Int64 2. When data gets aggregated in this processor, the mongo id is lost and becomes some sort of timestamp _id timestamp machingIdentifier processIdentifier counter time date timeSecond I wanted to keep the mongo id but it gets transformed to this. Any suggestions would help. Thank you
... View more