Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 108 | 12-17-2025 05:55 AM | |
| 169 | 12-15-2025 01:29 PM | |
| 115 | 12-15-2025 06:50 AM | |
| 242 | 12-05-2025 08:25 AM | |
| 403 | 12-03-2025 10:21 AM |
07-06-2018
03:53 PM
@Derek Calderon - Will need a developer with more custom processor development experience then me if this did not solve issue. I just know that adding additional jars/nars to NiFi's default lib dir can introduce class loader issues with regards to dependencies shared by multiple components. If issue still exists after clearing work directory and restarting, going to need a developer to suggest possible changes to your custom code. - Thanks Matt
... View more
07-06-2018
01:02 PM
@Derek Calderon - If you create a custom lib directory and place your new processor jar and its dependencies in there instead of NiFi's default lib directory, do you experience the same issue(s)? - It is not recommended that users added any new jars or nars to NiFi's default lib directory. Adding additional custom lib directories to NiFi is easy and also will make upgrading easier: Simply add a new property in the nifi.properties file: example: nifi.nar.library.directory.custom-lib1=/nifi/custom-lib1 - Then make sure the NiFi service user has proper ownership and permissions for this directory and your custom jars/nars you place in there. - A restart of NiFi is needed anytime you make a configuration change to nifi.properties or add a new jar/nar to any of the lib directories. - Before restarting NiFi, you should delete the existing NiFi "work" directory so it is recreated cleanly after this change - Thank you, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" on the answer.
... View more
07-06-2018
12:45 PM
1 Kudo
@Gitanjali Bare - When the NiFi Site-to-Site (S2S) capability was added to NiFi, the input and output ports were designed to allow for the movement of FlowFiles between NiFi process groups. Input and output ports allow for the movement of FlowFile to and from a parent process group. Once a FlowFile is on the "root" process group level, the effective parent would be another NiFi instance. S2S has been around a lot longer then NiFi's multi-tenant authorizations, thus at the time of development every user who could authenticate in to NiFi had access to everywhere on the NiFi canvas. - There is considerable design changes required to change this functionality in NiFi. The following Jira was opened as one suggested approach: https://issues.apache.org/jira/browse/NIFI-2933 - But in any design change, NiFi must consider how that design change will affect existing user during upgrade. The above change type may leave existing users with invalid flows after upgrade requiring potential substantial re-work. - At this time. the only option for NiFi S2S is having your input/output ports at the root canvas level (remote ports). - Another limitation in multi-tenant NiFi installations is authorized access to these remote input/output ports. The dataflows built by authenticated users do not execute as those users. All flows are executed as the NiFI service user. This means when NiFi A uses S2S to send/retrieve FlowFiles from NiFi B, the servers themselves are being authenticated and authorized in that connection. This means that once NiFI A has been authorized to see X number of remote ports on NiFi B, all users on NiFi A who add a Remote Process Group (RPG) pointing at NiFI B will be able to see and transfer FlowFiles to/from all those Remote ports. - Thank you, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.
... View more
07-06-2018
12:20 PM
1 Kudo
@umang s The following NiFi Expression Language statement will return "true" if a match is found: ${anyDelineatedValue("${temp_array}", ","):contains("${category}")} - Thanks, Matt - If you found this Answer addressed your original question, please take a moment to login and click "Accept" below the answer.
... View more
07-05-2018
12:22 PM
@Markus Wilhelm - The documentation states the following: Based on above you should be able to place your core-site.xml and hdfs.site.xml in the NiFi classpath. This will allow all HDFS based processors that do not have a configuration defined for this property to use those as the defaults. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.
... View more
07-03-2018
04:50 PM
This should be addressed by: https://issues.apache.org/jira/browse/NIFI-5065
... View more
07-03-2018
12:54 PM
1 Kudo
@philip carey - Ambari has built in variables for server hostnames. So in NiFi configs within Ambari you would configure the property as follows: The above will set the hostname properly to the correct unique hostname in the nifi.properties file on each NiFi node. - The hostname used should always be for a hostname assigned to the server where NiFi is running. - Thanks, Matt - *** Forum tip: Please try to avoid responding to an Answer by starting a new answer. Instead use the "add comment" to respond to en existing answer. There is no guaranteed order to different answers which can make following a response thread difficult especially when multiple people are trying to assist you.
... View more
07-03-2018
12:45 PM
1 Kudo
@Tommy - Why even use the "wait" and "notify" processors in this scenario? - All you need to do is feed the "success" relationship form location A PutFile processor to the processor writing to location B. Same file will be written to both locations and will only be written to location B if Location A was successful. - Thank you, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.
... View more
07-02-2018
08:21 PM
1 Kudo
@Saikrishna Tarapareddy - You can increase the number of concurrent tasks on each remote port you have established a connection to: Once you have your flow established/connected to your Remote Process Group (RPG), right click on the RPG to bring up teh following context menu: Select "Manage Remote Ports" to open a new UI. You will then be able to edit each Remote Input/Output you have established connection to: You will also want to increase the number concurrent tasks on your remote ports as well. - Thanks, Matt - When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.
... View more
07-02-2018
08:09 PM
@neeraj sharma *** Forum tip: Please try to avoid responding to an Answer by starting a new answer. Instead use the "add comment" tp respond to en existing answer. There is no guaranteed order to different answers which can make following a response thread difficult especially when multiple people are trying to assist you. - NiFi Process Groups have no logical binding to a "process". It is just a logical grouping of components on the canvas. Under the hood, NiFi simply has a collection of interconnected components that are being scheduled to execute based on each components configuration. There is no status based on a collection of components completing an operation on a specific FlowFile. So you would need to look at the status of individual processor components. - You may be able to extract the relevant information you are looking for by using the SiteToSiteBulletinReportingTask (get errors when failures occur) and/or SiteToSiteProvenanceReprotingTask (get info based on FlowFile events like create, drop, modify, etc..). - Thanks, Matt
... View more