Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi PutHDFS prepend filename with current date

Solved Go to solution
Highlighted

Nifi PutHDFS prepend filename with current date

Rising Star

Hello,

I am trying to read an API via NiFi at 30secs intervals and place the files to HDFS with filename.json prepended with the current date at each moment ('yyyy-mm-dd HH:mm:ss) but no success. I used GetHTTP processor along with UpdateAttribute to change the filename using expression language.

My attempt for that would be

${filename: prepend(${now():format("yyyy-MM-dd HH:mm:ss")})}

however i get a 'Relative path in absolute URI error' and the relationship fails. Otherwise if i remove the filename prepend operation everything works normal and the PutHDFS processor places the files in the specified folder correctly. Really appreciate any help, thanks in advance!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nifi PutHDFS prepend filename with current date

I'm pretty sure colons in filenames are considered bad practice - try removing that.


The dot could be because it's failing to complete the writing of the file, so it's generating the dot file for writing but can't complete the action. Just a guess without looking at your logs.

View solution in original post

4 REPLIES 4
Highlighted

Re: Nifi PutHDFS prepend filename with current date

@Foivos A

You're seeing that error because there is a blank space in the file name, try doing this:

${filename: prepend(${now():format("yyyy-MM-dd-HH:mm:ss")})}
Highlighted

Re: Nifi PutHDFS prepend filename with current date

Rising Star

@Wynner

Thanks for the suggestion, however i still get the same error with this expression

${filename: prepend(${now():format("yyyy-MM-dd-HH:mm:ss")})}

ERROR PutHDFS Failed to write to HDFS due to java.lang.IllegalArgumentException: java.net. URISyntaxException:

Relative path in absolute URI: .2017-08-01-11:01:13filename.json

I am not sure where this error comes from to be honest, plus there is a dot (.) before the date in the created filename in the error message that should not be there according to the expression used to prepend the filename.

Again, the flow is GetHttps --> UpdateAttribute --> PutHDFS and without any filename manipulations in the 2nd step everything works as it should. Any help is greately appreciated, thanks in advance!

Regards,

Foivos

Re: Nifi PutHDFS prepend filename with current date

I'm pretty sure colons in filenames are considered bad practice - try removing that.


The dot could be because it's failing to complete the writing of the file, so it's generating the dot file for writing but can't complete the action. Just a guess without looking at your logs.

View solution in original post

Highlighted

Re: Nifi PutHDFS prepend filename with current date

Rising Star

Thanks, colons were indeed the problem!

Don't have an account?
Coming from Hortonworks? Activate your account here