Member since
07-30-2019
3373
Posts
1616
Kudos Received
998
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
28 | 10-20-2025 06:29 AM | |
168 | 10-10-2025 08:03 AM | |
144 | 10-08-2025 10:52 AM | |
138 | 10-08-2025 10:36 AM | |
201 | 10-03-2025 06:04 AM |
10-20-2025
06:42 AM
@mansmaan I'm afraid there is not enough information shared yet to know exactly why you are experiencing a connection refused or connection reset issue from your NiFi node. I was able to use InvokeHTTP with no issue to successfully get a response from the URL you provided. I was able to view the weather data that was returned in json format. Have you tried to reach wttr.in directly from command line on the NiFi server? This sound like an environmental issue and not a NiFi invokeHTTP issue. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
10-20-2025
06:29 AM
@Rohit1997jio Yes it can be fast, but that depends on your dataflow(s). How many other dataflows is your NiFi also running? All these dataflows share the same JVM resources and utilize server resources. Also concurrency is important so having a match between number of partitions in your source Topic A and the concurrency within your ConsumeKafkaRecord processor is important. Concurrent tasks in the ConsumeKafkaRecord you be set using the following formula: (num partitions in Topic A) / (num NiFi nodes in cluster) = (num concurrent tasks set on ConsumeKafkaRecord) Example: Kafka Topic partions = 12 partitions NiFI cluster num nodes = 3 Concurrent tasks set on ConsumeKafkaRecord = 4 4 X 3 = 12 consumers in the consumer group (1 consumer per partition in source topic) Also in NiFI keep in mind that each concurrent task utilizes a thread from NiFi's "Max Timer Driven Thread Count" thread pool. The default pool size is 10. This means that only 10 concurrent task can execute at the same time. Generally speaking threads milliseconds, but for optimal performance you'll want to mange your server CPU load average resources and set the pool higher to maximize your throughput performance. NiFi can show you the Core Load Average (1 = 100% utilization of one physical core). So if your NiFi server has 4 cores and the load average is 4 or higher, the CPU is saturated. You'll be able to use this Core Load Average data to determine how much your can increase the size of the "Max timer Driven Thread Count" pool setting. NiFi recommends setting your "Max Timer Driven Count" initially to 2 - 4 times the number of Physical cores on the NiFi node. This assume all nodes in the NiFi cluster have same number of physical cores. If nodes have various numbers of physical cores, use the nodes with the fewest to set your initial pool size. Then monitor core load average across all NiFi nodes and adjust accordingly. There is no way to configure different thread pools per node in a NiFi cluster, NiFi expects all all servers in a NiFi cluster to have same hardware configuration. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
10-20-2025
05:58 AM
@pnac03 A MutualTLS connection is established between NiFi and NiFi-Registry. This means trust is established in both directions. You shared only the output from an openssl command connection to your NiFi-Registry. In that output we see that you NiFi-Registry reports the following: Acceptable client certificate CA names
CN = 3SCDemo-CA, OU = 3SCDemo
CN = nifi-registry, OU = 3SCDemo This tells us that your NiFi-Registry will only accept clientAuth certificates signed by the authorities listed in above output. Also from that same openssl command output we can see the following details of your NiFi-Registry's serverAuth certificate: Certificate chain
0 s:CN = nifi-registry, OU = 3SCDemo
i:CN = 3SCDemo-CA, OU = 3SCDemo
a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
v:NotBefore: Sep 13 18:35:02 2025 GMT; NotAfter: Sep 13 18:35:02 2026 GMT Here we can see that your NiFi-Registry certificate has been signed/Issued by the CA "CN=3SCDemo-CA, OU=3SCDemo". What you have not shared yet is details from the other side of this mutualTLS connection. If you ran the same openssl command pointing at your NiFi node it would show us some of the TLS connection details. You would want to look at the same NiFi's certificate chain and the NiFi's acceptable client certificate CA Names form that output. The CA names list must include the following to be able to trust the NiFi-Registry's serverAuth certificate: CN = 3SCDemo-CA, OU = 3SCDemo Then you want to look at the NiFi's certificate chain and make sure that all root and intermediate CAs in the trust chain exist in your NiFi-Registry's truststore. Currently your NiFi-Registry only trusts its own certificate and the one above CA. Also keep in mind the following: If you have a NiFi cluster setup, every node in the Nifi cluster must be able to establish a mutualTLS connection with your NiFi-Registry and not just the node URL you are currently connected to for UI access. If you do not have the following setting set to "TRUE" in the nifi.properties file, any changes made to the keystore and truststore are not loaded into NiFi until a NiFi restart: "nifi.security.autoreload.enabled". the default setting is FALSE. Reference: Security Configuration Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
10-20-2025
05:18 AM
@muahmmed We would need to know what is being logged in the nifi-registry-app.log during startup. Thank you, Matt
... View more
10-10-2025
08:03 AM
@Rohit1997jio For real-time streaming, Apache NiFi may not going to be the best option. NiFi works with files at rest. So you have an ingest processor that consumes from Kafka (consumed content is written to NiFi's content_repository and associated to a NiFi FlowFile that get moved to the outbound success relationship. ConsumeKafkaRecord processor likely to give better throughput). So even if you connect that success directly to a PublishKafka/PublishKafkaRecord processor, you'll still have a delay of some minimal amount of time as the Publish processor will get scheduled to execute and publish a FlowFiles message or record with multiple messages to the target cluster. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
10-08-2025
10:52 AM
@Frank168 Glad I was able to identify your issue for you. Can you accept the post that solved your issue. I see you accepted your response. Thank you, Matt
... View more
10-08-2025
10:51 AM
@Kiranq If your CPU load average is low relative to the number of cores you have on your Server running NiFi, you could increase the Max timer thread pool. A typical starting point is 2 to 4 times the number of cores. Also suggest running diagnostics (./nifi.sh diagnostics) and inspecting that output. Also check if you have any processors configured with a higher concurrent tasks value. Thanks, Matt
... View more
10-08-2025
10:44 AM
@Rashad_K What does NiFI provenance show for the duplicate inserts? Do you see the same NiFi FlowFile (by FlowFile UUID) being successfully inserted? Does the issue persist if your change the run duration from 25ms to 0ms? Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
10-08-2025
10:39 AM
@nifirequest This appears to be a duplicate to the following community question: https://community.cloudera.com/t5/Support-Questions/How-we-ignore-hostname-verification-certificate-for-minifi/td-p/412577 Did the response in that thread not help you? Thank you, Matt
... View more
10-08-2025
10:36 AM
@garb There is an existing Apache NiFi jira reporting this issue here: https://issues.apache.org/jira/browse/NIFI-14729 It aligns with your observations above. Thanks, Matt
... View more