Member since
03-10-2017
155
Posts
79
Kudos Received
32
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
293 | 08-12-2024 08:42 AM | |
776 | 05-30-2024 04:11 AM | |
1241 | 05-29-2024 06:58 AM | |
730 | 05-16-2024 05:05 AM | |
594 | 04-23-2024 01:46 AM |
03-28-2022
03:49 AM
1 Kudo
Answer to first question: you can use Reporting Task's to get processor status and enrich/trim the information as per you need with in NiFi or you can configure downstream queue for relationship Original from InvokeHTTP procesor and have count on that ,something similar on the lines https://community.cloudera.com/t5/Support-Questions/NiFi-Count-Fileflows-via-attribute/m-p/178860 Answer to Second question : NiFi stores processor related logs under nifi-app.log file which is by default located under /var/log/nifi/ , By default NiFi Processors only prints logs for start and stop operations and Warning and Error associated to the processors , user wont be seeing any events on what is being processed by processor until DEBUG is enabled for Processor class, which is only recommended to use in case of detailed information needed while troubleshooting any problem for short term period. You can filter specific component logs in nifi-app.log by using their component ID. Thank You.
... View more
03-14-2022
03:11 AM
1 Kudo
The error seems to be provided from nifi-boostrap.log , please check corresponding nifi-app.log and share for any warning or error leading to stop of nifi service so we can check further and help. Thank You.
... View more
03-14-2022
03:09 AM
Specific to NiFi : By default NiFi is not provided as docker image for CDH or CDP , Last docker image for NiFi was published though HDF NiFI https://docs.cloudera.com/HDPDocuments/HDF3/HDF-3.5.2/download/content/downloading-your-software.html that to be run standalone not managed by Ambari or Cloudera Manager. Thank you.
... View more
03-14-2022
02:59 AM
Do you see the same error string in nifi-app.log ? or there are any additional information in applogs ? if you see more details, please provide. Are you able to make any changes to flow design ? Can you please permissions and space is fine where flow.xml.gz is placed? Thank You.
... View more
03-02-2022
05:36 AM
For Site to Site , NiFi nodes to which data needs to be send or received should be part of polices , proxy and retrieve site-to-site details and send/receive data site-to-site.
... View more
02-28-2022
05:08 AM
1 Kudo
Few points when connecting to any DataBase from NiFi NiFi has Generic controller service named DBCPConnectionPool which used to create connection pool to given database and processors such as ExecuteSQL or PutSql will use this controller service to obtain connection and run the given queries. What is needed as minimum prerequisite to start with DBCPConnectionPool 1. Determine type of Database 2. Determine the JDBC URL syntax supported by Database 3. The Database client Driver which works outside of NiFi to connect to same Database needs to be placed on each NiFi nodes on given location which can be read user who is running NiFi as service All above needs to be provided by user , if Ucanaccess 5.0.1 client Driver works outside of NiFi then it should work with DBCPConnectionPool as well, Where you are getting this message "Given file does not exist" ? Is client driver is present on NiFi hosts and configured Database Driver Location is correct ? permissions? You mentioned There is a Cdata driver which works but that is a licensed product" , it works from where ? from NiFi:DBCPConnectionPool? Thank You.
... View more
01-24-2022
05:27 AM
Have you verified if class name "services.LDMService" is present in provided jar file?
... View more
01-24-2022
04:47 AM
1 Kudo
Same Command Arguments are you able to run as user nifi (or the user who is responsible to run nifi service )against path which is determine by fileslocation outside of nifi at command prompt ?
... View more
01-21-2022
03:15 AM
Not supported , as ListFTP or ListSFTP works on last modified timestamp to pick newly modified files since it ran before. So, if a file is added with older last modified timestamp than the one which Listftp already picked, then the file won't be picked with listftp logi, max or min file age property does not aline with current listing strategy.
... View more
01-21-2022
03:08 AM
In response to your queries : 1,do you konw how to add "Minimum File Age" property in listftp processor ? Ans : Not supported , as ListFTP or SFTP works on last modified timestamp to pick newly modified files since it ran before. So, if a file is added with older last modified timestamp than the one which Listftp already picked, then the file won't be picked with listftp logic. 2. listsftp can connect ftp server ? i try ,but failed Ans : Use ListFTP is FTP server is not secure. 3. What is the difference between listftp and listsftp? We have two type of FTP servers, FTP and SFTP , SFTP uses a secure channel to transfer files while FTP doesn't, thus NiFi has FTP and SFTP processors. To address your use case "file may be update anytime, so i need the "Minimum File Age" property like listsftp processor get" Ans : You need to change/update the logic on how files written at FTP server , If the same files getting updated/appended multiple time assuming write is not completed yet so try to rename the file with specific name pattern after append is completed and only list/fetch the files which matches with rename pattern using File Filter Regex settings.
... View more