Member since
08-14-2023
19
Posts
4
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1565 | 01-31-2024 02:21 AM |
10-22-2024
01:36 PM
1 Kudo
@edim2525 NiFi needs access to a lot of file handles since your dataflow can consist of a lot of components with multiples of concurrency plus you can have a lot of individual FlowFiles traversing your dataflows. The typical default open file limit is 10,000. I'd recommend setting a much larger open file limit of 100,000 to 999,999. This will solve your Too many open files error. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-31-2024
02:21 AM
1 Kudo
Upon examination, implementing the queue as a First In First Out prioritizer and configuring the load balancing strategy to Partition by attribute with the kafka.partition attribute has proven effective in maintaining the order.
... View more
01-31-2024
02:17 AM
1 Kudo
I'm trying to use CaptureChangeMySQL for mariadb but I'm getting one record that includes only metadata from the database (the table has 10 records ) {"type":"commit","timestamp":1706628370000,"binlog_filename":"mysql-bin.000001","binlog_position":516,"database":"copy"} I'm wondering if CaptureChangeMySQL can function with MariaDB?
... View more
Labels:
- Labels:
-
Apache NiFi
09-26-2023
07:48 AM
@MattWho My cluster is working with a single-user-authorizer . I tried your method and tested it on a running cluster with three nodes that configure as single-user-authorizer, I updated the three files (nifi.properties,login-identity-providers.xml,authorizers.xml) to work with LDAP configuration. When I restarted the first node (not primary or coordinator ), I got the following error messages in the log. 2023-09-26 11:20:34,441 ERROR [main] o.s.web.context.ContextLoader Context initialization failed
2023-09-26 11:50:19,381 ERROR [main] o.a.nifi.controller.StandardFlowService Failed to load flow from cluster due to: org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption.
2023-09-26 11:50:19,595 ERROR [main] o.a.n.c.c.node.NodeClusterCoordinator Event Reported for xxx:8443 -- Node disconnected from cluster due to org.apache.nifi.controller.serialization.FlowSynchronizationException: Failed to connect node to cluster because local flow controller partially updated. Administrator should disconnect node and review flow for corruption. The LDAP configuration takes effect only after restarting all the nodes
... View more
08-27-2023
11:53 PM
Hi All , I'm trying to replicate data between two Kafka topics that are defined with cleanup policy - compact using NIFI 1.21.0 ,The problem is that null messages(tombstones) replicate with a wrong value (no value instead of null ),I'm using PublishKafka_2_6 to insert the data to the target topic. for example message key 666 . source topic edi_test_compact . target topic edi_test_compact2 nifi
... View more
Labels:
- Labels:
-
Apache NiFi
08-20-2023
01:06 AM
I appreciate the comprehensive response, Thanks .
... View more
08-16-2023
06:57 AM
I set up the authorizers.xml file as you suggested and it's working perfectly, Thank you very much @MattWho !!
... View more