Member since
07-30-2019
3467
Posts
1641
Kudos Received
1018
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 151 | 05-06-2026 09:16 AM | |
| 245 | 05-04-2026 05:20 AM | |
| 236 | 05-01-2026 10:15 AM | |
| 467 | 03-23-2026 05:44 AM | |
| 352 | 02-18-2026 09:59 AM |
07-13-2021
12:38 PM
@Ash1 Not clear from yoru query how files are getting into or out of your NiFi. Assuming you have already received a set fo FlowFiles for the day in to your NiFi dataflow, the best approach may be to notify if any fails to be written out /transferred at the end of your dataflow. In this manor not only would you know that not all Files transferred, but would know exact what file failed to transfer. There are numerous processors that handle writing out FlowFile content (transfer) to another source or local file system. Those processing components typically have relationships for handling various types of failures. These relationships could be sent through a retry loop via the RetryFlowFile [1] processor back to the same transfer processor that failed. You define in the RetryFlowFile processor how many times you want a FlowFile to traverse this loop. After X number of loop it would get routed out of the loop to your PutEmail [2] processor where you could dynamically set the email content to include attributes from that FlowFile like filename, hostname of NiFi that failed to transfer it, etc... From the PutEmail processor you could send that FlowFile to somewhere else for holding until manual intervention was taken in response to that email. [`1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.RetryFlowFile/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.PutEmail/index.html If you found any of the response given here assisted with your query, please take a moment to login and click "Accept" on each of those solutions. Thank you, Matt
... View more
07-12-2021
01:27 PM
@LucasMorgado I don't see any ERROR? Was your old NiFi secured? Is the new NiFi secured? Along with your flow.xml.gz, did you also copy over your users.xml and authorizations.xml files to your new installation? Your screenshot indicates that your user is not authorized to see any of the components (Processors, Process groups, labels, etc), so they are all ghost implementations. Make sure your authorizers.xml is setup just like your old NiFi and that the above mentioned files were also copied over to your new install. NIFi only reads these files on startup. If you found this addressed yoru question, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-24-2021
05:34 AM
@midee I just ran the test myself with your sample as a one line formatted json and it still worked. {"expand": "renderedFields,names,schema,operations,editmeta,changelog,versionedRepresentations","id": "1155391","key": "DPGAUT-2","fields": {"customfield_10001": null,"customfield_10002": null,"customfield_10003": null,"customfield_10004": null,"customfield_10005": null,"customfield_10006": null,"resolution": null,"customfield_10101": "","customfield_10007": null,"customfield_10008": null,"customfield_10009": "This is required value","customfield_10010": null,"customfield_10011": null,"customfield_10012": null,"customfield_10013": null,}} Make sure your RouteOnContent processor dynamic property value does not have a line return in it. You'll notice it only shows "1" line above. If you see a "2" line then you have a line return in your Value field which would cause the issue you are seeing. Hope this helped, Matt
... View more
06-24-2021
05:22 AM
@Mahi123 Just to make sure i understand your use case correctly, you want to change the content only of your NiFi FlowFile? You have a NiFi FlowFile with following json content: {
"name": "Jon",
"title": "king of the nights watch"
} and you want to modify it to: {
"name": "Jon_<current date>",
"title": "king of the nights watch"
} I read your statement "I want to change the name of the file" as wanting to edit the filename and not the content of the file itself. Which is something totally different than above. Assuming you are looking to modify content and not the filename itself, you can do this using the ReplaceText processor after your EvaluateJsonPath processor. Search Value = ("name": "${name}") Replacement Value = "name": "${name}_${now():format('yyyy-MM-dd-HH-mm-ss')}" Replacement Strategy = Regex Replace Evaluation Mode = Line-by-Line NiFi typically has more than one way to satisfy the use case and there may be better option than the above using the various available record based processors. You may want to explore those as well. If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-18-2021
09:15 AM
@techNerd Questions for you: 1. Is Computer B consuming all the files LocalFolder and from every sub-directory within c:\users\LocalFolder? If yes, then your SFTP processor should be configured to Search Recursively. 2. Once consumed by computer B, are the original file being left on Computer A? If yes, you do NOT want to use the GetSFTP processor as it will refetch the same files over and over again each time it executes since they are not being removed from Computer A. A typical dataflow here would look like this: The ListSFTP processor would be configured to use as follows: The "Tracking Entities" listing strategy will help when source files may have older timestamps than files previously listed from a different directory. Using this listing strategy requires you to set a "Entity Tracking State Cache". There are multiple cache services to choose from. I simply used the "DistirbutedMapCacheClientService" controller service which gets configured to point at a "DistributedMapCacheServer"controller service I also setup within the same NiFi. The "Remote Path" is set to the top most folder you want to start listing files from ( C:\User\LocalFolder\ ). The "Search Recursively" property is set to "True" so that any random new sub folders added are also searched. The "Success" relationship is then routed via a connection from the ListSFTP to the FetchSFTP processor. The FlowFiles produced by the ListSFTP processor will have numerous FlowFile Attributes set on them that will be used later to persist your directory structure and to fetch the content via the FetchSFTP processor. You can list the flowfiles on a queue and click the "view details" icon next to nay listed FlowFile to see the attributes currently assigned to that FlowFile. For example: filename = testfile-32.txt
path = /tmp/LocalFolder
sftp.remote.host = <SFTP server hostname>
sftp.remote.port = 22 Then if you look at attributes for file from an added SubFolder: filename = testfile-31.txt
path = /tmp/LocalFolder/SubFolder
sftp.remote.host = <SFTP server hostname>
sftp.remote.port = 22 The FetchSFTP processor then uses these attributes set on each FlowFile to fetch the actual content from the source SFTP server: Downstream in your dataflow after you have fetched the content, you can still use the Attributes on each FlowFile when working with yoru FlowFiles. For example using the "path" attribute to dynamically control where a FlowFile will be written on a target system (might be the local NiFi via PutFile processor, another SFTP server somewhere else, etc.). If you found this addressed your query, please take a moment to login and click "Accept" on all provided solutions that helped. Thank you, Matt
... View more
06-17-2021
06:50 AM
1 Kudo
@techNerd Based on yoru example, you can actually modify your XML using the ReplaceText [1] processor. The processor can be configured with Java regex that would match on the string you want to remove and the replacement value would be set to an empty string. Your regex pattern in this case would need to match on (don't forget the spaces): ( unit = ".*?") The parenthesis mark the java regex capture group. .*? is a non greedy match on any character for zero or more characters until the very next quote. This allows this to match on any variety of unit strings ( meter, kilogram, and lb) from your example This will result in the following: <particular>
<name>Nicholas</name>
<height unit = "meter">183</height>
<weight unit = "kilogram" unit = "lb">75</weight>
</particular>
<particular>
<name>Debbie</name>
<height unit = "meter">163</height>
<weight unit = "kilogram" unit = "lb">45</weight>
</particular> being converted to: <particular>
<name>Nicholas</name>
<height>183</height>
<weight>75</weight>
</particular>
<particular>
<name>Debbie</name>
<height>163</height>
<weight>45</weight>
</particular> [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.ReplaceText/index.html If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-16-2021
06:26 AM
@techNerd I am looking at your example and am not clear on what your are "removing"? looks like you are adding text to your example. Thanks, Matt
... View more
06-16-2021
06:19 AM
@tech Please share a little bit more around your use case. The NiFi SFTP based processors are designed to create a FlowFile for each File retrieved from the target SFTP server and path. You want to retrieve a directory. NiFi has to write output from the retrieval to a FlowFile, so what are you looking to have written to that FlowFile? Then what do you want NiFi to do with folder once it is retrieved? More typically what is done in this situation is the ListSFTP [1]/FetchSFTP [2] (used when using NiFi cluster) or GetSFTP [3] (used when it is a standalone NiFi) processor are configured to retrieve all files from within a target folder on the SFTP server. One FlowFile is produced for each FlowFile that is retrieved. These processors all write FlowFile attributes to the produced FlowFiles. This includes the "path" attribute that tracks the directory/folder the files were fetched from. NiFi Expression Language (NEL) [4] is what is used to interact with these FlowFile attributes within your NiFi dataflow(s). Even if you did this outside of NiFi, you would get the folder and all files within it. Using above path attribute, each FlowFile knows folder name, so you could use that FlowFile attribute to write those FlowFile out to a target within the same original folder name. So same outcome can be accomplished. You can then do many things depending on your use case with these files once they are retrieved. Maybe use a mergeContent processor to create a tar file with all files added to it.... Maybe write these FlowFiles to a new target destination within a folder of same name as source... etc. [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.ListSFTP/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.FetchSFTP/index.html [3] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.GetSFTP/index.html [4] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html If this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-16-2021
05:17 AM
1 Kudo
@ubet You can run NiFi-Registry on Windows. The exception you are see is related to being unable to load a class that is included with NiFi-Registry. Maybe your download is no good. Did you try downloading the NiFi-Registry package again? Make sure you purge the work directory that was created by NiFi-Registry when the service attempted to start. Make sure you did not run out of disk space. You'll also want the product installation directory path and the directory path for the java JDK being used to NOT have spaces in the directory names. Use "Apache-nifi-1.12.1" instead of "Apache nifi-1.12.1". Reinstall your JDK outside of "Program Files". The above changes and checks should get you up and running successfully. If you found this addressed yoru query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-15-2021
06:29 AM
@Arash The "nifi.remote.input.socket.port" property is used so that the receiving NiFi can support the RAW transport protocol. This port has nothing to do with where within your dataflows the transferred data is being ingested. The Remote Process Group (RPG) that exists within your MiNiFi dataflow acts as a client. It will execute a background thread every 30 seconds that connects to the target NiFi URL configured in the RPG that fetches NiFi Site-To-Site (S2S) details from the target NiFi. These S2S details include details about the target. This information includes but is not limited to the following: 1. Hostnames of all nodes in target NiFi cluster 2. Remote (RAW) input socket port if configured 3. FlowFile load on each cluster node 4. Remote input and output ports this client is authorized access to. There is no option to configure multiple remote input socket port values. RPG would not know how to use them even if there was. In your case each unique MiNiFi should have be pushing to a different Remote input port on the canvas instead of all of them sending to same input port. Second option is to use a RouteOnAttribute processor after your single Remote input port that routes data based on the "s2s.host" attribute value set on the received FlowFiles. This attribute is set by the sending RPG on each MiNiFi to the hostname of that MiNiFi. You could of course also set a unique attribute on each FlowFile via na UpdateAttribute processor on each MiNiFi also before sending through the RPG. If you found this addressed your query, please take a moment to login and click "Accept" on solutions that helped you. Thank you, Matt
... View more