Support Questions

Find answers, ask questions, and share your expertise

NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

avatar
Explorer

I am trying to insert a flow file into MongoDb which has a createdAt date key record as an attribute.

When I inserted that flow file into MongoDb using PutMongo processor, it saves the "createdAt" attribute as a string. I want this to be saved as an ISO date object in mongoDB.

When I inserted flow file by sending "2020-08-26T04:00:00.000Z" PutMongo Processor inserts as a string.

When I sent like this “ISODate("2020-08-26T04:00:00.000Z”)” it also inserts same string as it is into mongo.

When I tried like this ISODate("2020-08-26T04:00:00.000Z”) without double quotation it throws error invalid object.

 

I need output as : 

{

“createdAt”: ISODate("2020-08-26T04:00:00.000Z”)

}

 

Kindly help if there is any way to do so.

1 ACCEPTED SOLUTION

avatar
Explorer

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

View solution in original post

4 REPLIES 4

avatar
Explorer

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

avatar
Expert Contributor

I also tried it with the Avro schema and type 'timestamp-millis' but I had the problem, that the milliseconds everytime got saved as .000 instead of .123, .987, ...

 

So another solution for you would be to use Jolt and add

"createdAt": { "$date": "${dateAttr}" }

to your JSON, that converts the type to date in MongoDB 

avatar
New Contributor

Put Mongo is throwing an error while using this 
***PutMongo[id=e57c5334-b5a8-1c6c-0840-73050fa4acd5] Failed to insert FlowFile[filename=a2268e35-1e97-4f75-b58c-f33396290666] into MongoDB due to Failed to parse string as a date: org.bson.json.JsonParseException: Failed to parse string as a date 
Caused by: java.time.format.DateTimeParseException: Text '2023-04-18 13:07:38.000+0000' could not be parsed at index 10****

Is there a way to use jolt and putMongo to store the iso date in mongo db 
pls help
thanks

avatar
New Contributor

"createdOn": {"$date":"${now():format('YYYY-MM-dd HH:mm:ss'):replace(' ', 'T'):append('.000+00:00')}"}
used this to store ISO date format in mongo db from Jolt and nifi expression language.