Member since
07-30-2019
3406
Posts
1623
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 314 | 12-17-2025 05:55 AM | |
| 375 | 12-15-2025 01:29 PM | |
| 355 | 12-15-2025 06:50 AM | |
| 345 | 12-05-2025 08:25 AM | |
| 594 | 12-03-2025 10:21 AM |
07-14-2022
09:12 AM
1 Kudo
@sayak17 so this looks doable as I described above. Not sure what program you are using in your screenshots as it does not show the full rest-api call structure. But your process would be as as follows: 1. Get bearer token 2. Extract bearer token from response body and place in FlowFile attribute 3. Format flowfile attribute as needed for use by the rest of the api-calls 4. Using bearer token make get request to the ../sites rest-api endpoint 5. Use ExtractText to extract the site id to a new FlowFile Attribute 6. Use bearer token again and use NiFi Expression Language to dynamically create the rest-api url to include the site id. 7. From response, extract the Drive id to another new FlowFile Attribute as above. 8. Use bearer token again and use NiFi Expression Language to dynamically create the rest-api url to include the drive id. 9 finally do whatever you need to do with the response json you get from that final rest-api call. Matt
... View more
07-14-2022
08:56 AM
@rafy Once you have created the certificates for your other two users as @DigitalPlumber suggested, you would need to connect to your NiFi as the admin user you setup during initial securing of your NiFi and these two new users via the NiFi global menu (upper right corner) --> Users. Then you would need to authorize those new user identities against any policies needed for them to perform the actions you want them to be allowed. https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies Hope this helps, Matt
... View more
07-14-2022
07:11 AM
@Meeran Sharing the full exception that was output when you tried to import key maybe helpful, But I am also not sure why you are trying to add a Client private key to your AWS load-balancer? I am not well versed on AWS load balancer configuration, so helping you get past your LB setup/configuration issue may be difficult for me. I do know TLS very well and explained the meaning of the two TLS exceptions you encountered. The HTTPS LB exception could be resolved by making sure the TrustedCertEntry(s) for the complete trust chain for the HTTPS LB's privateKey are present in the NiFi-Registry's truststore.jks. You should also add the trustedcertEntry for the NiFi CA you appeared to be using to the truststore used by your HTTPS LB. Thank you, Matt
... View more
07-14-2022
06:59 AM
@Meeran The users.xml is created/managed by the File-User-Group-Provider in the Authorizers.xml file. The authorizations.xml is created/managed by the File-Access-Policy-provider in the Authoirizers.xml file. Neither of these providers support using a database for persisting the users, groups, and authorizations. For more information on the authorization providers, follow the below link: https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#authorization NiFi-Registry supports using an embedded H2 DB (default) or an external DB (Postgres or MySQL) for storing knowledge of which buckets exist, which versioned items belong to which buckets, as well as the version history for each item. The actual version controlled flows are not stored in the DB https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#metadata-database The actual Version Controlled dataflow data is stored via the configured Persistence provider. The default provider is FileSystemFlowPersistenceProvider which writes this data to a directory locally on the NiFi-Registry host. The other available option is the GitFlowPersistenceProvider which commits this data to a remote Git repository when configured correctly. For more detail follow below link: https://nifi.apache.org/docs/nifi-registry-docs/html/administration-guide.html#persistence-providers If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
07-14-2022
06:44 AM
@Hashem1980 Neither the ConsumeIMAP or ConsumePOP3 NiFi processors support Oauth2 for authentication. NiFi does have an StandardOauth2AccessTokenProvider controller service. I would recommend opening an Apache NiFi Jira asking to have Oauth2 via this existing controller service integrated in to these processors. As an open source project contribution are always welcome. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
07-14-2022
06:31 AM
1 Kudo
@shashikumar NiFi only requires the the Java Runtime Environment (JRE). Latest versions of NiFi support both Java 8 or Java 11: system_requirements The JDK contains, in addition to the JRE, additional development tools like compilers, debuggers, etc. These may be useful to you depending on whether or not you intend to compile java code within your dataflow (not a requirement of any components like processors, controller services, reporting tasks, etc included with NiFi) or create your own custom NiFi components. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
07-14-2022
06:23 AM
@sayak17 Agree with @ckumar's response. It would be difficult to provide more detail without understanding the actual structure needed for the follow-on rest-api calls. What info is expected in headers versus raw data. Typically the access token is sent in a header and is often needed in all subsequent rest-api calls. Do you need to create unique FlowFile content to send as data? The ExtractText can be used to extract the token from the response written to the FlowFiles content and added as an attribute on the FlowFile before next invokeHTTP processor. Since I do not know much about Sharepoint or the format of the responses received, I can only give an example based on a returned token from NiFi's rest-api: The entire response body would be placed in a FlowFile attribute named "token". In the case of the NiFi token, I must then build a proper "Authorization" header with that token before being able to make additional rest-api calls using it. To do that, you can use an UpdateAttribute processor: So here I added "Bearer " to the token and write it to a new FlowFile attribute "Authorization"; although, you could also simply update the existing "token" attribute instead. Then in the next InvokeHTTP processor you can pass this "Authorization" FlowFile attribute value in a header by adding a dynamic property: Beyond this, you can do similar with your follow on responses assuming each step simply requires you to use the response values you extracted as headers or custom rest-api remote URLs. If you found any of the response received help you address your query, please take a moment to login and then click "Accept as Solution" on every response that helped. Hopefully this helps you reach your end goal here, Matt
... View more
07-11-2022
09:41 AM
1 Kudo
@Chakkara Updating a parameter context is a multi command process. 1. Fist you must GET the current parameter context configuration: curl -k -X GET https://<hostname>:<port>/nifi-api/parameter-contexts/<parameter context UUID> --compressed - This will return a potentially large json which will include the current revision, all the parameter contexts from that parameter context and all the referencing components. Assuming your NIFi is secured, your'll also need to include a method of authentication (client certificate, bearer token) in the above request. 2. Next you will POST your update-request using detail from above json response in step 1. curl -k 'https://<hostname>:<port>/nifi-api/parameter-contexts/<PARAMETR Context UUID>/update-requests' --data-raw $'{"revision":{"version":<num from step 1>},"id":"<PARAMETR Context UUID>","uri":"https://<hostname>:<port>/nifi-api/parameter-contexts/<PARAMETR Context UUID>","permissions":{"canRead":true,"canWrite":true},"component":{"name":"param-test","description":"","parameters":[{"parameter":{"name":"<Parameter name>","sensitive":false,"value":"<new parameter value>","description":""}}],"boundProcessGroups":[{"revision":{"clientId":"edcad62f-0181-1000-2fc6-de0c25282a3b","version":<num from step 1>},"id":"70b33f0a-e362-1c0b-8147-8fada18d56bd","position":{"x":136,"y":-2456},"permissions":{"canRead":true,"canWrite":true},"component":{"id":"70b33f0a-e362-1c0b-8147-8fada18d56bd","parentGroupId":"7930f131-017b-1000-0000-00007ece6b9d","position":{"x":136,"y":-2456},"name":"Param2","comments":"","variables":{},"parameterContext":{"id":"<PARAMETR Context UUID>","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"flowfileConcurrency":"UNBOUNDED","flowfileOutboundPolicy":"STREAM_WHEN_AVAILABLE","runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"inputPortCount":0,"outputPortCount":0},"runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"parameterContext":{"id":"994938a5-0180-1000-0000-00007cbba46b","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"inputPortCount":0,"outputPortCount":0},{"revision":{"clientId":"edcad62f-0181-1000-2fc6-de0c25282a3b","version":2},"id":"edcb3ff7-0181-1000-0000-00004abfd1ed","position":{"x":-323.1760214048029,"y":-2454.121877860687},"permissions":{"canRead":true,"canWrite":true},"component":{"id":"edcb3ff7-0181-1000-0000-00004abfd1ed","parentGroupId":"7930f131-017b-1000-0000-00007ece6b9d","position":{"x":-323.1760214048029,"y":-2454.121877860687},"name":"param1","comments":"","variables":{},"parameterContext":{"id":"994938a5-0180-1000-0000-00007cbba46b","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"flowfileConcurrency":"UNBOUNDED","flowfileOutboundPolicy":"STREAM_WHEN_AVAILABLE","runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"inputPortCount":0,"outputPortCount":0},"runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"parameterContext":{"id":"994938a5-0180-1000-0000-00007cbba46b","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"inputPortCount":0,"outputPortCount":0},{"revision":{"version":0},"id":"7930f131-017b-1000-0000-00007ece6b9d","position":{"x":328,"y":400},"permissions":{"canRead":true,"canWrite":true},"component":{"id":"7930f131-017b-1000-0000-00007ece6b9d","versionedComponentId":"136b68f9-1621-366e-8e8a-1db651ad8808","parentGroupId":"e377355c-017a-1000-2658-01d1a778332f","position":{"x":328,"y":400},"name":"Matt\'s PG","comments":"","variables":{},"versionControlInformation":{"groupId":"7930f131-017b-1000-0000-00007ece6b9d","registryId":"e97b2503-017a-1000-0000-00000c0a8abc","registryName":"nifi-sme-75.novalocal","bucketId":"19f7633f-5c1d-4d80-9d0d-d95c7e4885f8","bucketName":"new-bucket","flowId":"6add76b8-e56b-4d36-99e2-8aba4f5a9c33","flowName":"Matts-PG","flowDescription":"","version":2,"state":"LOCALLY_MODIFIED","stateExplanation":"Local changes have been made"},"parameterContext":{"id":"<PARAMETR Context UUID>","permissions":{"canRead":true,"canWrite":true},"component":{"id":"994938a5-0180-1000-0000-00007cbba46b","name":"param-test"}},"flowfileConcurrency":"UNBOUNDED","flowfileOutboundPolicy":"STREAM_WHEN_AVAILABLE","runningCount":39,"stoppedCount":109,"invalidCount":74,"disabledCount":7,"activeRemotePortCount":0,"inactiveRemotePortCount":2,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"inputPortCount":0,"outputPortCount":0},"runningCount":39,"stoppedCount":109,"invalidCount":74,"disabledCount":7,"activeRemotePortCount":0,"inactiveRemotePortCount":2,"versionedFlowState":"LOCALLY_MODIFIED","upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"parameterContext":{"id":"<PARAMETR Context UUID>","permissions":{"canRead":true,"canWrite":true},"component":{"id":"994938a5-0180-1000-0000-00007cbba46b","name":"param-test"}},"inputPortCount":0,"outputPortCount":0},{"revision":{"clientId":"edcad62f-0181-1000-2fc6-de0c25282a3b","version":2},"id":"373a39ad-0443-10ec-9e85-69445fefe75e","position":{"x":-320,"y":-2216},"permissions":{"canRead":true,"canWrite":true},"component":{"id":"373a39ad-0443-10ec-9e85-69445fefe75e","parentGroupId":"7930f131-017b-1000-0000-00007ece6b9d","position":{"x":-320,"y":-2216},"name":"param3","comments":"","variables":{},"parameterContext":{"id":"994938a5-0180-1000-0000-00007cbba46b","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"flowfileConcurrency":"UNBOUNDED","flowfileOutboundPolicy":"STREAM_WHEN_AVAILABLE","runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"inputPortCount":0,"outputPortCount":0},"runningCount":2,"stoppedCount":0,"invalidCount":0,"disabledCount":0,"activeRemotePortCount":0,"inactiveRemotePortCount":0,"upToDateCount":0,"locallyModifiedCount":0,"staleCount":0,"locallyModifiedAndStaleCount":0,"syncFailureCount":0,"localInputPortCount":0,"localOutputPortCount":0,"publicInputPortCount":0,"publicOutputPortCount":0,"parameterContext":{"id":"994938a5-0180-1000-0000-00007cbba46b","permissions":{"canRead":true,"canWrite":true},"component":{"id":"<PARAMETR Context UUID>","name":"param-test"}},"inputPortCount":0,"outputPortCount":0}],"id":"<PARAMETR Context UUID>"}}' --compressed - This will return the uri for this new update request. A background process will take care of stopping changing, validating and starting all referenced components for the update parameter context. Assuming your NIFi is secured, your'll also need to include a method of authentication (client certificate, bearer token) in the above request. 3. Third you can execute a GET using the uri returned in step 2 to check progress of the request. curl 'https://<hostname>:<port>/nifi-api/parameter-contexts/<PARAMETER CONTEXT UUID>/update-requests/<UPDATE REQUEST UUID form step 2 response>' -k --compressed - This will return a json that will include within it the "percentCompleted". You'll need to check this endpoint until it reports 100% complete. Assuming your NIFi is secured, your'll also need to include a method of authentication (client certificate, bearer token) in the above request. 4. Final step is a DELETE to remove the update request that is now completed. Failing to do this leaves this request in NiFi Heap memory until a NiFi restart. curl -X DELETE 'https://<hostname>:<port>/nifi-api/parameter-contexts/<PARAMETER CONTEXT UUID>/update-requests/<UPDATE_REQUEST UUID from step 2 response>' -k --compressed - Assuming your NIFi is secured, your'll also need to include a method of authentication (client certificate, bearer token) in the above request. An easy way to help understand the --data-raw input needed for your specific update-request is to use the developer tolls found in most browsers to inspect/capture that actual rest-api calls made when you execute the actual action via the NiFi UI. The browser developer tools will also allow you to right click on the triggering action and copy as curl. You can then paste that action as a curl command in a terminal window and execute it yourself. Reference: https://nifi.apache.org/docs/nifi-docs/rest-api/index.html - specifically looking at the section for "Parameter Contexts" If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
07-08-2022
12:36 PM
@Meeran Your issue in both exceptions is with the mutual TLS handshake and not really a configuration issue in NiFi or NiFi-Registry. And may not even be an issue with your LBs. Your first exception shared is telling you that the Registry-Client hostname configured in your NiFi was not found as a SAN in the serverAuth certificate returned by the target server. In order for the mutual TLS handshake to succeed there would need to be a matching hostname SAN entry in the certificate. The Second exception you shared is telling you that the truststore.jks does not contain the required complete trust chain needed to trust the private certificates DN. The privateKey used by the Server and Client will have an owner DN and Issuer DN. The Issuer DN is the authority that signed that certificate and must be found in the truststore. Some issuers are intermediate authorities meaning that also have an owner DN signed by another issuer authority. This is a trust chain. So your truststore would also need to to contain the trustedCertEntry for that intermediate's issuer as well. Once you reach the issuers certificate where both the owner an issuer DN match, you have reached the root authority of that trust chain. This is all part of the TLS spec and not specific to anything NiFi is enforcing itself. you can use openssl in most but not all to get the public certs for the authorities: openssl s_client -connect <hostname>:<port> -showcerts If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
07-08-2022
12:20 PM
1 Kudo
@rafy The PutFile processor is used to write the content of FlowFiles to the local file system. It can not be used to write files tp a remote system. The reason you are seeing no error is because the PutFile processor is creating that directory path using the ip address as a folder name on your local machine where NiFi is running and writing your FlowFile's content there. If you want to write a remote server, you would need to use a processor like the putSFTP processor assuming you have an SFTP server running on that remote server that your can successfully connect with. Another option would be to install NiFi on both servers and use NiFi's Site-to-SIte capability to send FlowFiles from one NiFi to another and then on the receiving NiFi use the putFile processor to write your FlowFile content to disk. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more