How to remove folder and everything inside that folder in Apache Nifi?

[#1]

How to remove folder and everything inside that folder in Apache Nifi?

[#2 · Accepted solution]

@adam chui

By using Execute Stream Command processor we can pass the argument as the directory path that need to delete and by using rm -rf command we can delete folder and everything inside the folder.

Example:-

I'm having del_nifi directory in tmp directory and nifi having access to delete this folder

[bash tmp]$ ll
drwxr-xr-x 2 nifi       hadoop        29 May 11 09:04 del_nifi
[bash tmp]$ cd del_nifi
[bash del_nifi]$ ll
-rw-r--r-- 1 nifi hadoop 20 May 11 09:04 6728259595979699

Flow:-

GenerateFlowFile Configs:-

Added new property called directory with /tmp/del_nifi with the value.

ExecuteStreamCommand Configs:-

Command Arguments

-rf;${directory} //we are passing directory attribute and -rf as arguments to rm command

Command Path

rm

Now ExecuteStreamCommand processor will delete the directory/subdirectories/files in the directory.
-

More info regarding rm command

[bash tmp]$ rm --help
-r, -R, --recursive   remove directories and their contents recursively
-f, --force           ignore nonexistent files and arguments, never prompt

[#3]

@adam chui

We are not able execute commands in ExecuteStreamCommand processor like

[bash] cat <filename>|grep <search string>
(or) 
[bash] ls |wc -l

But you can use QueryRecord processor and write the sql query(to filter or count..) then the query is going to be executed on the contents of the flowfile.

Take a look into this link for more details regarding Query Record processor.

[#4]

Thanks, it is a good answer, may I know how to run a command Execute Stream Command processor with pipeline "|" ?