About MattWho

MattWho · ‎03-07-2022

@nifideveloper You will need to use a NiFi Expression Language (NEL) statement to check if the date within the filename matches today's date. So first thing is isolate from the filename just the date portion and then compare it the output from NEL "now()" function output formatted in same way as source filename date pattern. Without a sample filename to work with, it would be difficult for me to provide an example NEL statement. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

MattWho · ‎03-07-2022

@richG Your observations are correct. The conversion script code does convert those properties. https://github.com/apache/nifi/blob/main/minifi/minifi-toolkit/minifi-toolkit-configuration/src/main/java/org/apache/nifi/minifi/toolkit/configuration/dto/RemotePortSchemaFunction.java I generated the following Apache NiFi jira: https://issues.apache.org/jira/browse/NIFI-9772 If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

Carlb · ‎02-04-2022

Late response but... Are you running in FIPS mode?

sachin_32 · ‎01-31-2022

hi @MattWho , i checked with advance option also but wrong result is coming can you check this formula is it ok or not? ${mydate:toDate('MM/dd/yyyy'):format('u'):lt('6'):ifElse("${literal('6'):minus(${mydate:toDate('MM/dd/yyyy'):format('u')}):multiply('86400000'):minus(${mydate:toDate('MM/dd/yyyy'):toNumber()}):format('EEE,MM/dd/yyyy')}","${literal('518400000'):minus(${mydate:toDate('MM/dd/yyyy'):toNumber()}):format('EEE,MM/dd/yyyy')}")}

MattWho · ‎01-31-2022

@rafy You'll want to read up on the documentation on the Apache MiNiFi page here: https://nifi.apache.org/minifi/index.html Since MiNiFi does not provide a UI from which you can construct a NiFi dataflow, you will need to build the dataflow that you will use on your MiNiFi using a NiFi installation. The converter toolkit is what you can then use to change your NiFi dataflow template into the necessary MiNiFi yaml file. You may also find these community posts helpful: https://community.cloudera.com/t5/Support-Questions/How-send-data-from-nifi-to-minifi-same-config/td-p/325183 https://community.cloudera.com/t5/Community-Articles/Ingesting-Log-data-using-MiNiFi-NiFi/ta-p/248154 If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

MattWho · ‎01-31-2022

@vk21 This question is not related to the original question in this post. I recommend starting a new question, so as to avoid confusion via a new conversation when this post already has an accepted solution. Thanks, Matt

rafy · ‎01-28-2022

Hello, I'm a learner & i would like to use the method you made mentioned here to collect logs in a remote server & send to Nifi. Please, can you put me through because i have been battling with how to build a msi before the real implementation. Thank you so much.

MattWho · ‎01-25-2022

@RonMilne I recommend taking your initial CSV record file and partitioning by the "salesRepId" in to multiple new JSON records. This can be accomplished using the PartitionRecord processor utilizing a CSVReader and a JsonRecordSetWriter. Your PartitionRecord processor configuration would look like this: Your CSVReader would be configured something like this (you'll need to modify it for your specific record's Schema: Note: Pop-out shows the "Schema Text" property and don't forget to set "Treat First Line as Header" property to "true" The JsonRecordSetWriter would need to be configured to produce the desired JSON record output format. However, just leaving default configuration will out put a separate FlowFile for each unique "SalesRepId". If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt

ebeb · ‎01-24-2022

Thanks @MattWho , actually found a way to filter/search process groups by name using the Summary option in the top right menu. This is very useful to find all the ETL pipelines once we give proper names and then by entering a part of the name we can show all matching Process Groups. Being a newbie I am trying to compare Streamsets UI to Nifi UI so I can work the same way. Streamsets provides an initial list of all ETL pipelines to filter by name etc. I guess if just after Nifi login if we saw two links: Summary and Canvas then users can click intuitively on the summary screen and review all their Process groups and click on specific PG they want to work with. This would make it similar to other ETL tools like Streamsets.

MattWho · ‎01-19-2022

@OliverGong I would avoid dataflow design when possible where you are extracting the entire contents of a FlowFile to FlowFile attribute(s). While FlowFile content only exists on disk (unless read in to memory by a processor during processing), FlowFile attributes are held in NiFi's JVM heap memory all the time (There is per connection swapping that happens when a specific connection reaches the swap threshold set in the nifi.properties file). FlowFiles with lots of attributes and/or large attribute values will consume considerable amounts of JVM heap which can lead to JVM Out Of Memory (OOM) exceptions, long stop-the-world JVM Garbage Collection (GC) events, etc... When options exist that avoid adding large attributes, those should be utilized. Thanks, Matt

Online	Online
Last Visited	‎02-04-2026 12:53 AM

Member Since	‎07-30-2019 10:41 AM
Last Visited	‎02-04-2026 12:53 AM
Posts	3,434
Kudos received	1628

Cloudera Community

Re: Setting TTL per key when writing to redis

Re: Best Practice for configuring registry flows

Re: Nifi 2.7.2 Start Problem

Re: Error importing NiFi workflow template from ve...

Re: nifi 2.6 registry security scan results

Re: NIFI RouteOn Atrribute unmatched flow files re...

Re: MiNiFi RPG Input Port batch settings not suppo...

Re: NiFi-1.14.0 failure to start: Unable to start ...

Re: how get next Week Sunday in NIFI

Re: Pull logs from a remote server to Nifi

Re: Unable to upgrade TLSv1.2 in Nifi 1.5

Re: How to collect windows event log using Nifi

Re: Sales Data Split by Sales Rep?

Re: NiFi - Understanding how to use Process Groups...

Re: How can I used JoltTransformJson Processor to ...