Member since
03-10-2017
155
Posts
79
Kudos Received
32
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
282 | 08-12-2024 08:42 AM | |
739 | 05-30-2024 04:11 AM | |
1224 | 05-29-2024 06:58 AM | |
720 | 05-16-2024 05:05 AM | |
587 | 04-23-2024 01:46 AM |
03-18-2024
04:33 AM
2 Kudos
I suspect one more NiFi service is running on the same host with the same port, It can be checked easily by running "ps -ef | grep nifi"
... View more
03-12-2024
06:22 AM
2 Kudos
@broobalaji HDF 1.8.0.3.3.1.0-10 was released way back in 2017. I strongly recommend upgrading to a much newer release of CFM. NiFi Templates have been deprecated and are completely removed as of Apache NiFi 2.x releases. Apache NiFi deprecated templates for a number of reasons: 1. Templates uploaded to NiFi (even if not instantiated/imported to the NiFi canvas reside within NiFi's heap memory space) 2. Large uploaded templates or many uploaded templates can have a substantial impact on NiFi performance because of the amount of heap they can consume. Simply increasing the size of NiFi's heap is also not the best solution to that heap usage as large heaps just lend themselves to longer stop-the-world garbage collections with the JVM. 3. Apache NiFi deprecated and moved away from using xml based flow in favor of json flow definitions around the Apache NiFi 1.16 time frame. Flow definitions (JSON files) can exported and imported without uploading them in to heap memory within NiFi. The above info aside.... It is best to use the developer tools available in your web browser to inspect/capture the rest-api call being made when you perform the same steps directly via the NiFi UI. This makes it easy to understand the calls that need to be made in your automation. I also encourage you if you continue to use templates to upload, import to UI, and then delete the uploaded template to minimize heap impact. Thanks, Matt
... View more
01-29-2024
05:44 AM
@PriyankaMondal Just to add to What @ckumar provided, the NiFi repositories are not locked to the specific node. What i mean by that is that they can be moved to a new node, withe "new" being the key word there. A typical prod NiFi setup will use protected storage for its flowfile_repository and content_repository(s) which hold all the FlowFile metadata and FlowFile content for all actively queued and archived FlowFiles on a node. To prevent loss of data, these repositories should be protected through the use of RAID storage or some other equivalent protected storage. The data stored in these repositories is tightly coupled to the flow.xml.gz/flow.json.gz that cluster is running on every node. Let's say you have hardware failure, it may be faster to standup a new server then repair the existing hardware failure. You can simple move or copy the protected repositories to the new node before starting it. When the node starts and joins your existing cluster it will inherit the cluster's flow.xml.gz/flow.json.gz and then begin loading the FlowFiles from those moved repositories in to the connection queues. Processing will continue exactly where it left off on the old node. There is no way to merge repositories together, so you can not add the contents of one nodes repositories to the already existing repositories of another node. The provenance_repository holds lineage data, and the database_repository holds flow configuration history and some node specific info. Neither of these are needed to preserve the actual FlowFiles. Hope this helps, Matt
... View more
01-03-2024
08:15 AM
@PriyankaMondal What is being logged in the nifi-user.log when the issue happens? Have you tried using your browser's developer tools to look at the data being exchanged in the request with the NiFi cluster? Feels like maybe the site cookies are not being sent to the NiFi node after successful authentication resulting in the exception being seen. Thanks, Matt
... View more
12-28-2023
05:23 AM
Looking at the thread, Ans is No, that's not the way NiFi Processor is meant to handle relationships, The Client Processor has to initialize /start properly with the given configuration then only it can try to write and then based on the write results flow files can be routed to failure or success, Here Processor can't even start properly.
... View more
12-08-2023
02:03 AM
@RenoldSingh, have any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
09-03-2023
10:42 PM
1 Kudo
Update : This is working as I hoped for With this configuration the FileProcessor group will take the next flowfile only after completely processing the flowfile that is inside the group. Thank you @SAMSAL, @pvillard
... View more
08-30-2023
06:02 AM
2 Kudos
InvokeHTTP does not provide a way to read user Authentication in the form of secrets from files or environment variables.
... View more
08-06-2023
06:07 PM
2 Kudos
You need to look at CDC tools like Cloudera Kafka Connect with Debezium or Cloudera SQL Streaming Builder with Debezium. Kafka Connect https://docs.cloudera.com/runtime/7.2.17/kafka-connect/topics/kafka-connect-connector-debezium-oracle.html SSB https://docs.cloudera.com/csa/1.10.0/how-to-ssb/topics/csa-ssb-cdc-connectors.html Or you can do Oracle GoldenGate with Kafka. I have some examples in this article https://medium.com/cloudera-inc/cdc-not-cat-data-capture-e43713879c03
... View more
06-30-2023
06:25 AM
@wallacei That is correct. NiFi is not part of CDP Base. You need a trial for CFM.
... View more