how to use mongodb aggregate querys in get mongo processor in nifi

[#1]

Hi

Is there any possibility to use mongodb aggregate query in getmongo processors . If not what is the alternatives Please suggest

I am using mentioend query in get mongo processor. This query rran with out any issues in mongo db . while supplying htis query in nifi gave below error

db.Pdtata.aggregate( {$match: {project:"ABC"}}, {$group: {_id : {d:"$projectid",c:"$caseid"}, subtotal:{$sum:1}} }, {$group: {_id:"$_id.d", caseid:{$push:{caseid:"$_id.c", subtotal:"$subtotal"}}, total:{$sum:"$subtotal"}} } )

error

00:09:00 EDT ERROR 1b581f0a-0164-1000-9009-b7a133d91f95

GetMongoFailed to execute query Document{{$match=Document{{project:ABC}}}} due to Query failed with error code 2 and error message 'unknown top level operator: $match' on server  com.mongodb.MongoQueryException: Query failed with error code 2 and error message 'unknown top level operator: $match' on 

[#2 · Accepted solution]

As of NiFi 1.6.0, there is a processor called RunMongoAggregation that should do what you want. I don't think you'll need the "db.Pdata.aggregate" part, just the JSON query body, possibly as an array (i.e. the Mongo shell may know to treat that list of objects as an array but the NiFi processor may expect a valid JSON array of objects).

[#3]

Hi Matt,

I am trying to use the RunMongoAggregation processor and ran into 2 issues.

1. I ran an aggregation in my mongo very similar to this:

[ { $project: { lastUpdated: { $dateFromString: { dateString: '$lastUpdated' } } } } ]

It changes the "lastUpdated" value from a string type to a DateTime type. The result looks like this:

lastUpdated 2018-07-23T10:15:05Z DateTime

I tried this in the RunMongoAggregation and with double quotes around each field as this link suggests https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-mongodb-nar/1.6.0/org.apache.... But the result I got looks like this:

lastUpdated NumberLong("1532340905000") Int64

2. When data gets aggregated in this processor, the mongo id is lost and becomes some sort of timestamp

_id

timestamp

machingIdentifier

processIdentifier

counter

time

date

timeSecond

I wanted to keep the mongo id but it gets transformed to this.

Any suggestions would help. Thank you