Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 301 | 12-17-2025 05:55 AM | |
| 362 | 12-15-2025 01:29 PM | |
| 330 | 12-15-2025 06:50 AM | |
| 338 | 12-05-2025 08:25 AM | |
| 585 | 12-03-2025 10:21 AM |
09-01-2021
12:22 PM
1 Kudo
@SAli12 Please share how you have your MergeContent processor configured. If I am following your query correctly, you have a source FlowFile that you pass through a SplitJson processor which produce a number of output FlowFiles. Later after processing all those individual FlowFile you want to combine them all back in to a single FlowFile so that you can generate only a single FlowFile which gets passed to a putEmail processor. Correct? If that is the case, the SplitJson will create a set of Attributes on each split FlowFile it creates: Within either the MergeContent or MergeRecord processor you have the option to set the "Merge Strategy" to "Defragment". This strategy uses those attributes on the FlowFile to bin files base on the fragment.identifier and makes sure that all files are present before a merge happens***. Note: if all bins are being utilized by different fragment identifiers and another unique identifier comes in, the oldest bin will be processed and sent to failure since it is missing all it fragments. But above should get you the one FlowFile your are looking for instead of multiple FlowFiles with partial merges of merges of your original source file. If you found this response addressed your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
09-01-2021
12:07 PM
@wbivp Is the exact user string you see in the nifi-user.log the same (case sensitive) as what is set in Ranger? is this user string authorized for the /flow policy? Thank you, Matt
... View more
09-01-2021
12:04 PM
@wbivp The nifi-user.log output you shared indicates that the user string "cdpadmindev@DOMAIN" has not been authorized to against the NiFi /flow resource identifier (View the user interface). The authorizers.xml configuration file controls how user and/or group based authorizations are setup and managed. So first things is what is configured in that file? Is it using a file based authorizer or external Ranger based authorizer? Is it using any user group providers? Did you configure and initial admin identity? if so, does that initial admin identity string exactly match your user string from the nifi-user.log you shared? If using the file based authorizer, you should have in the authorizers.xml both the "file-user-group-provider" and the "file-access-policy-provider". These providers are used to create the users.xml and authorizations.xml file on startup if they do NOT already exist. so even if you do have the initial admin set correctly, if NiFi was started previously before the user string was set, you would have existing users.xml and authorizations.xml files without this user and the required admin policies set. (remove these files and restart NiFi so they are generated again). If using Ranger as your authorizer, you need to make sure that the user string exactly as you see in the nifi-user.log exists as a user in Ranger and the the NiFi service plugin in ranger is setup and has the correct NiFi resource identifier policies authorized for that user. Here is a reference article on those Ranger based NiFi policies: https://community.cloudera.com/t5/Community-Articles/NiFi-Ranger-based-policy-descriptions/ta-p/246586 If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
09-01-2021
11:47 AM
1 Kudo
@Justee ListSFTP only generate a FlowFile with attributes/metadata about the file on the SFTP processor. It does not look at the content itself. So your filtering options are limited to what is in those generated attributes. The FetchSFTP processor uses these attributes/metadata to retrieve the actual content and add it to the existing FlowFile produced by the ListSFTP processor. So unfortunately you would need to fetch the all files and then keep on those that contain the desired value in the third field. You may want to look at the RouteText [1] processor for handling these Files after they are the content is fetched. [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.14.0/org.apache.nifi.processors.standard.RouteText/index.html If you found this response addressed your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
08-24-2021
10:59 AM
@smartraman This can also be accomplished through a different and more complex configuration of the ReplaceText processor: Using below input content example: {
"TOT_NET_AMT" : "[\"55.00\"]",
"H_OBJECT" : "File",
"H_GROSS_AMNT" : "[\"55.00,58.00\"]",
"TOT_TAX_AMT" : "[9.55]"
} I would set up the replaceText processor as follows: Instead of just searching for those character patterns and replacing them with nothing, I break entire input line-by-line in to a series of capture groups. That way I can omit the capture groups matching the patterns you want removed ([ or [\" or \"] or ]) and then manipulate the capture group containing a possible comma separated list, so that only the last value in that list is returned. I used below java regular expression which results in 5 capture groups: (.*?)([\Q[\E]\\\"|[\Q[\E])(.*?)(\\\"[\Q]\E]|[\Q]\E])(.*?)$ I then used the following Replacement Value in which I used NiFi expression language against the 3rd capture group. If that capture group does not contain any commas, the entire string is returned. With example above and this configuration, you end up with the following new content: {
"TOT_NET_AMT" : "55.00",
"H_OBJECT" : "File",
"H_GROSS_AMNT" : "58.00",
"TOT_TAX_AMT" : "9.55"
} If you found this helped with your latest query, please take a moment to login and click on "Accept as Solution" below this response. Thank you, Matt
... View more
08-24-2021
09:46 AM
@midee @guima The OAuth2TokenProviderImpl controller service was added to NiFi around the 1.13 release timeframe. At the time of its creation, there were no processors within NiFi yet that use it natively. It's only usable in scripted processors that customers would use. However the Apache NiFi community is working on ways to leverage this controller service in processors such as InvokeHTTP in the future. You are encourage to participate and contribute to the Apache NiFi product. Hope this helps, Matt
... View more
08-13-2021
12:36 PM
@mickt Correct.... This recent bug fix is fixed in the current master and will be part of the next Apache NiFi 1.15 release.
... View more
08-13-2021
12:26 PM
@dupuy_gregory Can you share the configurations for both your GetHDFS processors?
... View more
08-13-2021
12:20 PM
@SolidSnake Site-To-Site is not a proxy. Perhaps a diagram of what you are trying to accomplish may make it easier to follow your use case? Could you share such a diagram that shows flow of connections and what is happening now versus a diagram that shows what you want to happen. You have a 2 node nifi cluster using invokeHTTP processor making a request to a handleHttpRequest processor on the standalone NiFi (1 node cluster) and then the FlowFile generated by the HandleHttpRequest processor is end via Site-To-Site to the NiFi cluster that then routes to a HandleHttpResponse? Maybe i am not following very well, but I don't understand why you are doing that if I understand your flow correctly? Why isn't the same node that receives the the request via the HandleHttpRequest processor also sending the response via a HandleHttpResponse processor? Thanks, Matt
... View more
08-13-2021
08:16 AM
1 Kudo
@mickt I looks as though you may be hitting: https://issues.apache.org/jira/browse/NIFI-8938 This issue was introduced via a change that was introduced as part of NiFi 1.14: https://issues.apache.org/jira/browse/NIFI-8723 If you found this response addressed your query, please take a moment to login and click "Accept as Solution". Thank you, Matt
... View more