Member since
12-03-2017
148
Posts
25
Kudos Received
11
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1182 | 11-03-2023 12:17 AM | |
2911 | 12-12-2022 09:16 PM | |
1104 | 07-14-2022 03:25 AM | |
1769 | 07-28-2021 04:42 AM | |
2035 | 06-23-2020 10:08 PM |
08-01-2022
12:34 AM
Unfortunately dont find anything. I enabled DEBUG logger also for HandleHttpRequest processors but nothing over there
... View more
07-31-2022
10:25 AM
Hello Experts, I have HandleHttpRequest processors which works fine when I start my load test and after 100K + records suddenly stops taking any more request (does not emit any ff also from processor) for 10-15 seconds and then again resumes ! Any idea what could be the reason, it does not show any log also. Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-29-2022
07:29 PM
Thank you @MattWho . - 0 flow file in outbound connection. - no backpressure - nifi 1.12.1 version we are using - Don't see any logs related to this - We are using 500 gb disk and only 20gb was utilized at that time But all repos (flowfile, content and prov) are located in same ebs volume of 500gb, so is this could be something due to lack of iops availability?
... View more
07-28-2022
12:28 AM
Hello Experts, We have HandleHttpRequests processor running with 16 concurrency. Container queue size configured is 1000. While running a load test suddenly HandleHttpRequests stopped emitting flowfiles recieved and started throwing error show in below image - So we stopped the load test and now even after 30 mins when we send requests it will just emit 2 or 3 flow files and again shows same error and does not release/emit any request flowfiles. Ideally when no request is getting hit to that port (which HttpRequestListener is configured) container should get cleared and flow files should get created from queue and emit right? What could be the reason this processor is just trhowing container full error and not emitting any flow files ? It started working only when we manually cleanup all repositories (content, provenance, ff) and restart the nifi ! Any suggestion would be appreciated. Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-25-2022
09:40 PM
Experts, I am continuously seeing this log in Nifi app log - 2022-07-25 06:12:40,788 INFO [Timer-Driven Process Thread-219] o.a.n.c.repository.FileSystemRepository Unable to write to container default due to archive file size constraints; waiting for archive cleanup Any idea what is the reason and will there be any impact of this ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-14-2022
03:25 AM
testRunner.setAllowSynchronousSessionCommits(true); Helped me to get rid of this error
... View more
07-13-2022
01:14 AM
Hello Experts, I am using org.apache.nifi.util.TestRunners in Nifi custom processor unit test cases, and code something like below.. testRunner = TestRunners.newTestRunner(O9QueueRouting.class); . . . . MockFlowFile ff = this.testRunner.enqueue(new ByteArrayInputStream(ffContents.getBytes(StandardCharsets.UTF_8))); Map<String, String> attrs = new HashMap<String, String>(); attrs.put("routingheader", "headervalue"); attrs.put("kafka.topic", "topic1"); ff.putAttributes(attrs); testRunner.run(); It was working earlier now started giving error like below - [ERROR] O9QueueRoutingTest.test:73 java.lang.RuntimeException: As of version 1.14.0, ProcessSession.commit() should be avoided when possible. See JavaDocs for explanations. Instead, use commitAsync(), commitAsync(Runnable), or commitAsync(Runnable, Consumer<Throwable>). However, if this is not possible, ProcessSession.commit() may still be used, but this must be explicitly enabled by calling TestRunner. Any idea how this can be resolved ? How to explicitly enable by calling TestRunner ? Thanks Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-09-2022
05:24 AM
It depends on your use case - how many tables, how much records each table has, one time migration vs cdc etc. For one time migration (for till date) , if data is huge then its better to run sql paginated query instead of full table select so that resulting flow file will not be very huge and hence no impact on nifi performance. If you have multiple tables then may be you can add a generate flow file processor for triggering the flow with all the tables names in json and split the json and send each table name to ExecuteSQLQuery processor to execute and get data.. If CDC need migrated then may be you can schedule the flow very frequently and query the data based on datetime range if possible or any incremental values (which can be cached using nifi cache for new run)
... View more
07-09-2022
04:20 AM
Hello Experts, I am upgrading nifi from 1.12.x to 1.16.x and for some time I need to support both the versions. So I am trying to have profiles for nifi 12 and 16 in maven pom so that based on the profile passed it can build the nar file for specific version. I am facing an challenge in above step, because of a refactoring happened in nifi security utils post 1.12. I was using 'OkHttpClientUtil' in 1.12 and which is no longer available in 1.16 So now to support both the version for a custom processor I need to have 2 java classes, one for 1.12 and one for 1.16 with minor code changes. But I want to keep my custom processor name same across both the version, is it possible ? Ex : my java classes like below ... Nifi 12 - ResourceRouting.java Nifi 16 - ResourceRoutingV2.java Is it possible to have custom processor name as "ResourceRouting" ResourceRoutingV2.java also? Any suggestion would be much appreciated. @araujo @MattWho @GangWar @smdas Thanks, Mahendra
... View more
Labels:
- Labels:
-
Apache NiFi
07-05-2022
10:50 PM
1 Kudo
Issue created : https://issues.apache.org/jira/browse/NIFI-10197
... View more