Created 06-20-2018 04:17 AM
Hi
Is there any possibility to use mongodb aggregate query in getmongo processors . If not what is the alternatives Please suggest
I am using mentioend query in get mongo processor. This query rran with out any issues in mongo db . while supplying htis query in nifi gave below error
db.Pdtata.aggregate( {$match: {project:"ABC"}}, {$group: {_id : {d:"$projectid",c:"$caseid"}, subtotal:{$sum:1}} }, {$group: {_id:"$_id.d", caseid:{$push:{caseid:"$_id.c", subtotal:"$subtotal"}}, total:{$sum:"$subtotal"}} } )
error
00:09:00 EDT ERROR 1b581f0a-0164-1000-9009-b7a133d91f95
GetMongoFailed to execute query Document{{$match=Document{{project:ABC}}}} due to Query failed with error code 2 and error message 'unknown top level operator: $match' on server com.mongodb.MongoQueryException: Query failed with error code 2 and error message 'unknown top level operator: $match' on
Created 06-20-2018 03:41 PM
As of NiFi 1.6.0, there is a processor called RunMongoAggregation that should do what you want. I don't think you'll need the "db.Pdata.aggregate" part, just the JSON query body, possibly as an array (i.e. the Mongo shell may know to treat that list of objects as an array but the NiFi processor may expect a valid JSON array of objects).
Created 06-20-2018 03:41 PM
As of NiFi 1.6.0, there is a processor called RunMongoAggregation that should do what you want. I don't think you'll need the "db.Pdata.aggregate" part, just the JSON query body, possibly as an array (i.e. the Mongo shell may know to treat that list of objects as an array but the NiFi processor may expect a valid JSON array of objects).
Created 07-25-2018 03:12 PM
Hi Matt,
I am trying to use the RunMongoAggregation processor and ran into 2 issues.
1. I ran an aggregation in my mongo very similar to this:
[ { $project: { lastUpdated: { $dateFromString: { dateString: '$lastUpdated' } } } } ]
It changes the "lastUpdated" value from a string type to a DateTime type. The result looks like this:
lastUpdated 2018-07-23T10:15:05Z DateTime
I tried this in the RunMongoAggregation and with double quotes around each field as this link suggests https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.6.0/org.apache.... But the result I got looks like this:
lastUpdated NumberLong("1532340905000") Int64
2. When data gets aggregated in this processor, the mongo id is lost and becomes some sort of timestamp
_id
timestamp
machingIdentifier
processIdentifier
counter
time
date
timeSecond
I wanted to keep the mongo id but it gets transformed to this.
Any suggestions would help. Thank you