Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

Solved Go to solution

NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

New Contributor

I am trying to insert a flow file into MongoDb which has a createdAt date key record as an attribute.

When I inserted that flow file into MongoDb using PutMongo processor, it saves the "createdAt" attribute as a string. I want this to be saved as an ISO date object in mongoDB.

When I inserted flow file by sending "2020-08-26T04:00:00.000Z" PutMongo Processor inserts as a string.

When I sent like this “ISODate("2020-08-26T04:00:00.000Z”)” it also inserts same string as it is into mongo.

When I tried like this ISODate("2020-08-26T04:00:00.000Z”) without double quotation it throws error invalid object.

 

I need output as : 

{

“createdAt”: ISODate("2020-08-26T04:00:00.000Z”)

}

 

Kindly help if there is any way to do so.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

New Contributor

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

View solution in original post

2 REPLIES 2
Highlighted

Re: NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

New Contributor

I have achieved this by using AVRO schema.Screen Shot 2020-08-29 at 9.06.06 PM.png

In place of putMongo Now I am using putMongoRecord and convertRecord for converting schema. 

Here is an example for AVRO schema : 

{
"type": "record",
"namespace": "nifi",
"name": "fredSchema",
"fields": [
 {
    "name": "createdAt",
    "type": {
      "type": "int",
      "logicalType": "date"
    }
 }
]
}

Use the above AVRO schema, and remember the date what you are going to sen should be yyyy-MM-dd format. Thanks. 

View solution in original post

Highlighted

Re: NiFi Processor: Save ISO format date into mongoDb using PutMongo Processor

Contributor

I also tried it with the Avro schema and type 'timestamp-millis' but I had the problem, that the milliseconds everytime got saved as .000 instead of .123, .987, ...

 

So another solution for you would be to use Jolt and add

"createdAt": { "$date": "${dateAttr}" }

to your JSON, that converts the type to date in MongoDB 

Don't have an account?
Coming from Hortonworks? Activate your account here