Support Questions
Find answers, ask questions, and share your expertise

How to get Nifi to use a timestamp as a directory name?

How to get Nifi to use a timestamp as a directory name?

Explorer

I have Nifi taking a number of json files from Kafka. I set up a "ConsumeKafka" to start the process. Now I need to read the json file's "creationDate" field, which is a normal UNIX timestamp (i.e., 2019-01-19T04:34:28.527722+00:00). It then needs to take part of that date and format it so it matches a corresponding HDFS directory, so the files can go into a directory that matches the date - i.e., the directories are named "01-19-2019", "01-20-2019", "01-21-2019", etc.

I was thinking of setting up an "EvaluateJsonPath" processor with a property of "creationDate" and a value of "$.creationDate". Then there would be an "UpdateAttribute" processor with a property of "creationDate" and a value of "${creationDate('yyyy/MM/dd HH:mm:ss:SSS'Z'):format('MM-dd-yyyy')}" Finally there would be a "PutHDFS" processor with a directory of "/${creationDate}".

I'm not sure about the expression (or if this is going to work the way I think it is). Especially because the directory format doesn't quite match the timestamp format. Maybe the directory names can change, but I'm going to assume that they can't. Any thoughts on how I can make this work?