question Re: Trying to move files via ExecuteStreamCommand in Support Questions

Trying to move files via ExecuteStreamCommand

noncitizen — Mon, 03 Apr 2023 15:22:32 GMT

First attempt:

Command Arguments = ${absolute.path:append('*.log')};${absolute.path:append('logfiles')}

Command Path = /bin/mv

No working directory set

Execution error = /bin/mv: cannot stat â/path/input/*.logâ: No such file or directory

So I ran "/bin/mv /path/input/*.log /path/input/logfiles" from putty — no errors, files moved.

Second attempt:

Based on some online troubleshooting, e.g., https://unix.stackexchange.com/questions/77007/mv-cannot-stat-no-such-file-or-directory-in-shell-script

Same as first, but first argument changed to "${absolute.path}"*.log

Execution error = same as above, except double quotes around first path

Third attempt:

Command Arguments = *.log;logfiles

Command Path = /bin/mv

Working Directory = ${absolute.path}

Execution error = /bin/mv: cannot stat â*.logâ: No such file or directory

So in putty, I cd into /path/input and type "/bin/mv *.log logfiles" — no errors, files moved.

Many folks online having the same/similar problem say "my bad, I had the wrong path" or "my bad, the files aren't there". No, for me, the path is right, and the files are there. Attribs on files/folders are 777.

I'm using ExecuteStreamCommand all over the place with /bin/rm to delete files without any problem. This is the first time I'm trying to move files using /bin/mv.

At first, the special character was tripping me up, but then this (below) led me to believe it was not relevant to the problem, just a problem with Linux reporting the error using smart quotes and NiFi losing it in translation.

https://stackoverflow.com/questions/18081038/bad-character-showing-up-in-bash-script-execution

It takes a lot for me to reach out for help, so I'm hoping I'm not missing something really obvious.

Re: Trying to move files via ExecuteStreamCommand

VidyaSargur — Tue, 04 Apr 2023 07:11:43 GMT

@noncitizen, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @cotopaul @SAMSAL @TimothySpann @MattWho who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: Trying to move files via ExecuteStreamCommand

cotopaul — Tue, 04 Apr 2023 13:43:56 GMT

hi @noncitizen,

I have tested the following configuration on my local machine and it seems to be working fine. You can give it a try and let me know if it works:

Using GenerateFlowFile, I created a FlowFile having the attribute "absolute.path" defined as "/home/test/testFolder/". Now, you already have a flow which generates you all these information so you can skip this step.

In ExecuteStreamCommand, I have defined the following:
Command Path = bash
Command Arguments = -c;mv ${absolute.path:append('*.log')} ${absolute.path:append('logfiles')}

The result is that all the files with the extension .log from within my folder /home/test/testFolder/ have been copied into /home/test/testFolder/logfiles.
I need to mention though the following:
- the absolute path should end with "/", otherwise, using your expression you will have something like /path/input*.log.

Re: Trying to move files via ExecuteStreamCommand

noncitizen — Tue, 04 Apr 2023 13:57:06 GMT

Bingo! Thanks so much.