Skip to main content
Manoj90
Gold ⭐️
Aug 26, 2020 11:07 AM UTC
Solved

NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

I am trying to insert a flow file into MongoDb which has a createdAt date key record as an attribute.

When I inserted that flow file into MongoDb using PutMongo processor, it saves the "createdAt" attribute as a string. I want this to be saved as an ISO date object in mongoDB.

When I inserted flow file by sending "2020-08-26T04:00:00.000Z" PutMongo Processor inserts as a string.

When I sent like this “ISODate("2020-08-26T04:00:00.000Z”)” it also inserts same string as it is into mongo.

When I tried like this ISODate("2020-08-26T04:00:00.000Z”) without double quotation it throws error invalid object.

 

I need output as : 

{

“createdAt”: ISODate("2020-08-26T04:00:00.000Z”)

}

 

Kindly help if there is any way to do so.

Manoj90
Manoj90Answer
Gold ⭐️
Aug 29, 2020 5:22 PM UTC

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
{
"name": "createdAt",
"type": {
"type": "int",
"logicalType": "date"
}
}
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

4 replies

Manoj90
Manoj90AuthorAnswer
Gold ⭐️
Aug 29, 2020 5:22 PM UTC

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
{
"name": "createdAt",
"type": {
"type": "int",
"logicalType": "date"
}
}
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

Translate
Faerballert
Gold ⭐️
Sep 3, 2020 3:18 PM UTC

I also tried it with the Avro schema and type 'timestamp-millis' but I had the problem, that the milliseconds everytime got saved as .000 instead of .123, .987, ...

 

So another solution for you would be to use Jolt and add

"createdAt": { "$date": "${dateAttr}" }

to your JSON, that converts the type to date in MongoDB 

Translate
ramMuka
Silver ⭐️⭐️
Apr 19, 2023 5:27 AM UTC

Put Mongo is throwing an error while using this 
***PutMongo[id=e57c5334-b5a8-1c6c-0840-73050fa4acd5] Failed to insert FlowFile[filename=a2268e35-1e97-4f75-b58c-f33396290666] into MongoDB due to Failed to parse string as a date: org.bson.json.JsonParseException: Failed to parse string as a date 
Caused by: java.time.format.DateTimeParseException: Text '2023-04-18 13:07:38.000+0000' could not be parsed at index 10****

Is there a way to use jolt and putMongo to store the iso date in mongo db 
pls help
thanks

Translate
ramMuka
Silver ⭐️⭐️
Apr 27, 2023 4:54 AM UTC

"createdOn": {"$date":"${now():format('YYYY-MM-dd HH:mm:ss'):replace(' ', 'T'):append('.000+00:00')}"}
used this to store ISO date format in mongo db from Jolt and nifi expression language.

Translate