Member since
07-30-2019
3398
Posts
1621
Kudos Received
1001
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 483 | 11-05-2025 11:01 AM | |
| 374 | 11-05-2025 08:01 AM | |
| 596 | 11-04-2025 10:16 AM | |
| 734 | 10-20-2025 06:29 AM | |
| 874 | 10-10-2025 08:03 AM |
02-21-2017
03:09 PM
1 Kudo
@mayki wogno You can reduce or even eliminate the WARN messages by placing a MergeContent processor between your first and second DeleteHDFS processors that merges using "path" as the value to the "Correlation Attribute Name" property. The resulting merged FlowFile(s) would still have the same "path" that would be used by the second DeleteHDFS to remove your directory. Matt
... View more
02-21-2017
02:10 PM
That was the intent... It would only be successful after all files where deleted first. So only after the last file was removed would the directory deletion be successful.
... View more
02-21-2017
01:48 PM
@mayki wogno FlowFiles generated by the listHDFS processor all have a "path" attribute created on them: That attribute could be used to trigger you directory deletion via the DeleteHDFS processor. What is difficult here is determining when all data has been successfully pulled from an HDFS directory before deleting the directory itself. You could try using two DeleteHDFS processors in series with one another. The first DeleteHDFS deletes the files from the target "path" of the incoming FlowFiles and the second deletes the directory (Recursive property set to false). Matt
... View more
02-21-2017
01:25 PM
@Pradhuman Gupta The WARN and ERROR messages you see when you float your cursor over the red notification icon on a processor are also written to the nifi-app.log. There is no way to capture those bulletins directly from a processor and route them to a putEmail processor. If there are specific processor types for which you want to monitor for WARN and/or ERROR messages for, you could modify your NiFi's logback.xml file so that logs generated by those processors classes are written to their own output log file. You could then setup a dataflow that tails that new log and sends an email when WARN and/or ERROR log messages are written to it. Thanks, Matt
... View more
02-21-2017
01:16 PM
@mayki wogno Make sure the user your NiFi is running as is authorized to delete files and directories in your target HDFS. The DeleteHDFS processor properties are as follows: Thanks, Matt
... View more
02-16-2017
05:11 PM
1 Kudo
@Anshuman Ghosh The Search Value Regex above has 4 capture groups from a valid IP address. Each capture group can then be referenced in the replacement Value as $1, $2, $3, and/or $4. In the example above the replacement for each found valid IP is still the first two numbers followed by ".x.x". You can of course change the replacement value to whatever meets your specific needs. Thanks, Matt
... View more
02-16-2017
05:06 PM
1 Kudo
@Anshuman Ghosh I posted your question here for you.
... View more
02-16-2017
05:03 PM
1 Kudo
Hello, We have a requirement to anonymize IP address. So we would identify IP address and replace the last part with some arbitrary value. We have tried using the following regex for that (with "Regex Replace" for replacement strategy, "Line-by-Line" as evaluation mode) (1) \\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3} (2) ^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$ (3) ((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?) None of this works. Can you help me with the right value of regex we can use and the replacement value as well. Thank you in advance!
... View more
Labels:
- Labels:
-
Apache NiFi
02-16-2017
04:02 PM
@mayki wogno Setting /tmp will cause listHDFS to produce a listing of files in all 4 of your directories. Following that listing, use a routeOnAttrinute processor to auto-terminate andy listing that were not from /tmp/toto or /tmp/truc before feeding what FlowFiles are left down the rest of your dataflow.
... View more
02-16-2017
03:44 PM
@marksf Correct, the "key" icon will allow you to apply component level access policies to your process group with id "863928d5-12e7-...." Once "view the component" is granted the ID will be replaced with the actual process group name.
"Modify the component" will allow the user to also add, modify and delete components within that process group. Thanks, Matt
... View more