Member since
06-18-2018
39
Posts
0
Kudos Received
0
Solutions
10-21-2022
12:56 PM
@rangareddyy What is important to understand is that the NiFi component processors are not being executed by the user authenticated (assuming secured NiFi) into NiFi, but rather by the NiFi service user. So let's say that your NiFi service is owned by a "nifiservice" linux account. Whatever umask is configured for that user will be applied to directories and files create by that user. Now if your script is using sudo, it is changing the user that executes your script resulting in different user ownership and permission from the "nifiservice" user. Subsequent component processors will also execute as the "nifiservice" user and then not have access to those files and directories. So you'll need to take this in to account as you built your scripts. Make sure that your scripts are adjusting permissions on the directory tree and files as needed so your "nifiservice" user or all users can access the files needed downstream in your dataflows. So in yoru case it sounds like your script executed by ExecuteScript processor is creating a sh file not owned by the "nifiservice" user or does not have execute permission set on it. The ExecuteStreamCommand processor will attempt to execute the sh command on disk as the "nifiservice" user only. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
07-26-2022
08:36 AM
At moment you can not run stored procedure using ExecuteSQL, however, you can use ExecureScript to do the same with stored procedure wrapped in script with MySQL engine or any other client you can use please refer : https://community.cloudera.com/t5/Support-Questions/Does-ExecuteSQL-processor-allow-to-execute-stored-procedure/m-p/158922 If you found this response assisted with your issue, please take a moment to login and click on "Accept as Solution" below this post. Thank you
... View more
07-20-2022
08:23 AM
I think you will also need to do the transfer/commit in the each list logic
... View more
08-24-2018
04:03 AM
@Rangareddy Y you can take help of file filter.
... View more
08-21-2018
11:17 AM
@Steven Matison /homepath/customer_*/inbox/ is giving error. processor is invalid and getting directory does not exist error.
... View more
08-21-2018
06:52 AM
Thanks for the reply @Harald Berghoff can you suggest me how to give the input directory path in listFile processor for below scenario. lets assume these folders exists in local system. /homepath/ranga/inbox/ /homepath/krishna/inbox/ /homepath/kaja/inbox/ /homepath/jilebhi/inbox/ /homepath/laddu/inbox/ /homepath/bundhi/inbox/ like 100+ folders are there...i have used multiple listFile processor based on folder size files. lets assume, 3 listFile processors i used in my flow and want to configure perticular folders in listFile processor. in first listFile processor, i want to point (ranga, krishna) folder paths, in second listFile processor, i want to point (kaja, jilebhi) folder paths, in third listFile processor, i want to point (laddu, bundhi) folder paths, can you please suggest me how to achieve above case... Thanks, Rangareddy Y
... View more
08-03-2018
05:16 AM
can you send me the job runner code exact line with comment
... View more
06-29-2018
05:54 AM
@Bryan Bende Can you suggest me how to implement below scenario in nifi. 500+ partners used to send files to us thru SFTP. how to read all these sftp connections for files in nifi. if listSFTP does not support dynamic parameter change.. what is the alternate solution for this. Can you please give me flow of steps.. Thanks, Rangareddy Y
... View more
06-19-2018
04:41 PM
2 Kudos
ListSFTP writes two attributes on each flow file - filename and path. In FetchSFTP you want to set the remote file to be a dynamic expression that uses these attributes: ${path}/${filename}
... View more
06-18-2018
12:49 PM
1 Kudo
@Rangareddy Y - NiFi does not support dynamic property creation from an XML based input file. Some properties on some processors will support NiFi's Expression Language (EL) which would allow supporting properties to retrieve a defined variables value. Those key/value variables can be defined in a number of places: 1. On the incoming FlowFiles to that NiFI processor in the form of FlowFile Attributes. 2. On the NiFi Process Group that contains your processors 3. From a NiFi variable registry file. (key/value property file. Cannot be updated in real time. changes to this file would require a NiFi restart) 4. NiFI JVM properties defined in NiFi's bootstrap.conf file (Restart required each time this file is edited) 5. NiFI process owner's environment variables (Restart of NiFi Required if user env variables are edited/added) - None of NiFi's processors will support the use of EL to define processor sensitive property values like passwords. Floating over the small "question mark" icon to the right of a property name in a processor will indicate if the use fo EL is supported by that processor configuration property. - Referenced documentation on NiFi's EL: https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#custom_properties - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.
... View more