Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

avatar
Explorer

I am trying to insert a flow file into MongoDb which has a createdAt date key record as an attribute.

When I inserted that flow file into MongoDb using PutMongo processor, it saves the "createdAt" attribute as a string. I want this to be saved as an ISO date object in mongoDB.

When I inserted flow file by sending "2020-08-26T04:00:00.000Z" PutMongo Processor inserts as a string.

When I sent like this “ISODate("2020-08-26T04:00:00.000Z”)” it also inserts same string as it is into mongo.

When I tried like this ISODate("2020-08-26T04:00:00.000Z”) without double quotation it throws error invalid object.

 

I need output as : 

{

“createdAt”: ISODate("2020-08-26T04:00:00.000Z”)

}

 

Kindly help if there is any way to do so.

1 ACCEPTED SOLUTION

avatar
Explorer

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

View solution in original post

4 REPLIES 4

avatar
Explorer

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

avatar
Expert Contributor

I also tried it with the Avro schema and type 'timestamp-millis' but I had the problem, that the milliseconds everytime got saved as .000 instead of .123, .987, ...

 

So another solution for you would be to use Jolt and add

"createdAt": { "$date": "${dateAttr}" }

to your JSON, that converts the type to date in MongoDB 

avatar
New Contributor

Put Mongo is throwing an error while using this 
***PutMongo[id=e57c5334-b5a8-1c6c-0840-73050fa4acd5] Failed to insert FlowFile[filename=a2268e35-1e97-4f75-b58c-f33396290666] into MongoDB due to Failed to parse string as a date: org.bson.json.JsonParseException: Failed to parse string as a date 
Caused by: java.time.format.DateTimeParseException: Text '2023-04-18 13:07:38.000+0000' could not be parsed at index 10****

Is there a way to use jolt and putMongo to store the iso date in mongo db 
pls help
thanks

avatar
New Contributor

"createdOn": {"$date":"${now():format('YYYY-MM-dd HH:mm:ss'):replace(' ', 'T'):append('.000+00:00')}"}
used this to store ISO date format in mongo db from Jolt and nifi expression language.