Trying to move files via ExecuteStreamCommand

Question

First attempt:Command Arguments = ${absolute.path:append('*.log')};${absolute.path:append('logfiles')}Command Path = /bin/mvNo working directory setExecution error = /bin/mv: cannot stat â/path/input/*.logâ: No such file or directory So I ran "/bin/mv /path/input/*.log /path/input/logfiles" from putty — no errors, files moved. Second attempt:Based on some online troubleshooting, e.g., https://unix.stackexchange.com/questions/77007/mv-cannot-stat-no-such-file-or-directory-in-shell-scriptSame as first, but first argument changed to "${absolute.path}"*.logExecution error = same as above, except double quotes around first path Third attempt:Command Arguments = *.log;logfilesCommand Path = /bin/mvWorking Directory = ${absolute.path}Execution error = /bin/mv: cannot stat â*.logâ: No such file or directory So in putty, I cd into /path/input and type "/bin/mv *.log logfiles" — no errors, files moved. Many folks online having the same/similar problem say "my bad, I had the wrong path" or "my bad, the files aren't there". No, for me, the path is right, and the files are there. Attribs on files/folders are 777. I'm using ExecuteStreamCommand all over the place with /bin/rm to delete files without any problem. This is the first time I'm trying to move files using /bin/mv. At first, the special character was tripping me up, but then this (below) led me to believe it was not relevant to the problem, just a problem with Linux reporting the error using smart quotes and NiFi losing it in translation. https://stackoverflow.com/questions/18081038/bad-character-showing-up-in-bash-script-execution It takes a lot for me to reach out for help, so I'm hoping I'm not missing something really obvious.

cotopaul · Accepted Answer

hi @noncitizen,I have tested the following configuration on my local machine and it seems to be working fine. You can give it a try and let me know if it works:Using GenerateFlowFile, I created a FlowFile having the attribute "absolute.path" defined as "/home/test/testFolder/".  Now, you already have a flow which generates you all these information so you can skip this step.In ExecuteStreamCommand, I have defined the following:Command Path = bashCommand Arguments = -c;mv ${absolute.path:append('*.log')} ${absolute.path:append('logfiles')}The result is that all the files with the extension .log from within my folder /home/test/testFolder/ have been copied into /home/test/testFolder/logfiles.I need to mention though the following:- the absolute path should end with "/", otherwise, using your expression you will have something like /path/input*.log.

VidyaSargur · Answer

@noncitizen, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @cotopaul @SAMSAL @TimothySpann @MattWho who may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.