Member since
04-09-2019
25
Posts
1
Kudos Received
0
Solutions
07-20-2022
08:40 AM
Hello All, I'm attempting to convert from a GetHDFS processor in apache nifi in anticipation of it being deprecated. I am going to use the ListHDFS instead yet when I set up the same inputs it is only returning the file path in a flow file and not actual folder content(files). Can anyone help with this? I'm just sourcing from a HDFS directory.
... View more
Labels:
- Labels:
-
Apache NiFi
12-13-2019
07:02 AM
@fowler7878 You are correct that MonitorActivity can not be used to monitor on activity based on per month thresholds. Your use case is not one NiFi is designed to handle. NiFi is designed to work with FlowFiles and typically NiFi processor are not designed to return directory listings. Your use case may require you to build your own custom NiFi processor or perhaps your own custom script that your can execute via ExecuteStreamCommand or ExecuteScript processors where the script returns number of files in a target directory which you can then make notification routing decisions with. I was just trying to offers a builtin solution/suggestion while not exactly what you are looking for. Thank you, Matt
... View more
12-11-2019
11:06 AM
@fowler7878 No problem. At this point you have extracted the the count successfully and used updateAttribute processor to subtract 31 from it. You do not need to write it back to the content to use it in putEmail processor. The majority of the putEmail processor configuration properties support NiFi Expression Language also. You can retrieve the value of your FlowFiiles attribute with this simple NiFi EL statement: ${COUNTFILES} So you can use above anywhere in the string you configure in the the "Subject" and/or "Message" configuration properties. Thanks, Matt If you found my answer help you to a solution, please don't forget to mark it as the accepted solution.
... View more
05-07-2019
01:30 PM
1 Kudo
If you are not obtaining keys from the database, not using fragmented transactions, and not rolling back on failure, then you should see the failed flow files in a batch being routed to the failure relationship. If you must configure the processor differently, then the flow files will be treated as a single transaction. In that case, in order to handle individual failures you'll want to not use batches, meaning set PutSQL's Batch Size property to 1.
... View more
04-20-2019
09:04 PM
@James Fowler We need to use ReplaceText processor after GenerateTableFetch processor and replace select * with: select col1, UCR_COST_IN_$ as UCR_COST_IN from table //replace * with column names and add an alias
... View more