Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

NiFi RunMongoAggreation - How to get all documents within a time range?

New Contributor

I am trying to get all documents within a time range using RunMongoAggreation.

In console I tried with below query and it worked fine.

db.collection.find(
{ "header.timestamp": {
        $gte: ISODate("2018-12-20T05:19:35.904Z"),
        $lt: ISODate("2018-12-20T05:21:44.509Z")
    } }
)

Flowfile has below properties

startTimestamp: 2018-12-20T05:19:35.904Z

endTimestamp: 2018-12-20T05:21:44.509Z

Below is the query being used in RunMongoAggreation

{
  "header.timestamp": {
    "$gt": {
      "$date": "${startTimestamp}"
    },
    "$lt": {
      "$date": "${endTimestamp}"
    }
  }
}

Its not working gettting below exception

com.fasterxml.jackson.databind.exc.mismatchedinputexception cannot deserialize instance of `java.util.arraylist` out of START_OBJECT token
2 REPLIES 2

New Contributor

Convert field to timestamp then compare.

 

[
    {
        "$addFields": {
            "timestamp": {
                "$toLong": "$header.timestamp"
            }
        }
    },
    {
        "$match": {
                "timestamp": {
                    "$gte": ${startTimestamp:toDate("yyyy-MM-dd HH:mm:ss"):toNumber()}
                },
                "timestamp": {
                    "$lte": ${endTimestamp:toDate("yyyy-MM-dd HH:mm:ss"):toNumber()}
                }
         }
    },
    {
        "$project": {
            "timestamp": 1
        }
    }
]

 

New Contributor

Hello! 

Try to use flow UpdateAttribute - GetMongo.

 

To UpdateAttribute create attr with value 

 

{
  "header.timestamp": {
    "$gt": {
      "$date": "${startTimestamp}"
    },
    "$lt": {
      "$date": "${endTimestamp}"
    }
  }
}

 

 

To GetMongo in field Query insert name attr from prev step. 

 

RunMongoAggreation used for aggregation query, but select all docs is not effective, I think.