Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Dynamically update Date Format in RecordReader

Highlighted

Dynamically update Date Format in RecordReader

New Contributor

I've implemented the pattern found in this post for updating date formats in input files in a NiFi flow. This is working for a test case, but my broader goal is to be able to set the Date/Time/Timestamp Format to be used by the CSV Reader at run-time.

I have inbound files from many external sources, and the input date/time formats will be known for each file passing through the flow in advance (i.e. dd/MM/yyyy, MM/dd/yyyy hh:mm:ss, yyMMdd, etc.), so I can attach that to the flow as attributes somewhere upstream. There may be dozens of permutations of date, time, and timestamp across all files though. Let's assume for now that all files adhere to the same avro schema, with the exception of the date/time formats.

The properties "Date/Time/Timestamp Format" in the CSV Reader controller don't appear to accept expressions however. Am I stuck with either creating a CSV Reader controller for each permutation of formats and branching the flow, or is there another way to achieve this with the reader controller?

Screenshot of what I'm trying to dynamically change is attached.

Cheers

68452-screen-shot-2018-04-12-at-172011.png

Don't have an account?
Coming from Hortonworks? Activate your account here