Is there a way to use the Nifi Expression Language instead of traditional Regex to get the timestamp from an event of this format? 2017-05-04 14:43:17,302 foo bar foo bar I would assume I'd need something that could find the second white space, take everything before it and save it as an attribute and then I could easily substring that attribute into its parts. ... View more

So here is our setup. Server 1: TailFile -> PublishKafka Server 2: ConsumeKafka -> ExtractText -> Update Attribute -> MergeContent -> UpdateAttribute (create filename) -> PutHDFS We currently have it set up to parse out the timestamp from the files and save them as variable using the ExtractText command so we can create our filename and HDFS directories with the variables in this format: Examples: May_03_16_39 (May 3rd at 16:39 pm) May_03_16_40 (May 3rd at 16:40 pm) May_03_16_41 (May 3rd at 16:41 pm) Our directory structure goes down to the minute: 2017/May/03/16/39 What we see is that during the file rollover, it puts a few seconds of data from the end of one file and a few seconds from the beginning of the next file into a file called: May_03_16_ Please see screenshots of file structure output, PutHDFS config, UpdateAttribute (create filename) config and if you could use anything else that would help let me know. We are using the append function with PutHDFS to put all files of the same minute into a specific file. ... View more