Member since
11-03-2023
26
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1481 | 11-08-2023 09:29 AM |
08-27-2024
05:52 AM
Hi , In My nifi flow i have two dataFlows i want to same same data to two different HTTP endPoints , for both data flow data is being consumed from same kaflka topic (TOPIC A) both flows has different kafka groupID and pushing data to different HPPT end points (endPoint 1 and endPoint 2). If there is any failure data is being produced into retryTopic R which is common for both data flow . There is separate retryData flow for both configured with seperate HttpEndPoints , but both consuming from same retryTopic . What issue coming is , due to same retry topic , dataFlow A can consume data of dataFlowB . How should i avoid this situation ? retryTopic used should be same for both. Here is a sample for the dataFlow .
... View more
Labels:
- Labels:
-
Apache NiFi
12-27-2023
10:46 PM
My "error" topic is on different Kafka cluster . So i wanted my data which is failing due to connection error should move to failure and then to error topic . Is there any way to do so ??
... View more
12-26-2023
09:05 AM
In my flow iam publishing data to kafka topic , if there is failure , that request should go to failure relation where iam publishing that data to error topic , but in my case when there is not connectivity with kafka or kafka broker value is not correct request is getting queue in PublishKafka processor not moving to failure relation . This queue will effect my other requests any even my error handling logic .Please suggest some solution Attaching the image of error and my nifi flow-
... View more
Labels:
- Labels:
-
Apache NiFi
12-21-2023
10:40 PM
nifi version iam using is old Powered by Apache NiFi - Version 1.8.0.3.3.0.0-165 , i have developed the same solution using retryFlowFile processor , but its working in my other environment due to old nifi version , so i need to have same implementation in that too . @MattWho even retry relation is not there in this version. i was trying to achieve this by expression language , but that is also not working.
... View more
12-21-2023
11:53 AM
I want to retry my flow file if it has some error for 24 hours , retry should will happen every hours . if the file is there in the flow for more then 24 hours it should be removed .
iam trying achieve same by using update attribute processors , routeOnAttribute processor and taking help of expression language . Still iam not able to achieve it.
these are the expression I am using to check elapsed time and received Time of flow file in routeOnAttribute processor but its going to unmatched relation every time -
these values are for testing pursose -
elapsed_time = ${now():toNumber():minus(${queuedtimestamp:toNumber()}):divide(60000):gt(10)}
receivedTime= ${now():toNumber():minus(${receivedTime:toDate("yyyy-MM-dd HH:mm:ss"):toNumber()}):gt(180000)}
NOTE: retryFlowFile processor in not available as my nifi version is old
Please suggest better approach to do so .
... View more
Labels:
- Labels:
-
Apache NiFi
12-20-2023
01:32 AM
control rate processor is nor working
... View more
12-19-2023
12:54 PM
Hi , iam retrying a flow file 3 times before moving moving it to failure relationship , with each retry i want to have 10 seconds gap in between them , so 2nd retry will happen after 10 seconds . iam not able to understand how to use wait processor for this . What should be the configurations of that processor . iam using older version of nifi so i can not use retryFlowFile processor .
... View more
Labels:
- Labels:
-
Apache NiFi
12-14-2023
10:14 PM
Hi , is there any idle connection timeout for ConsumeKafka and PublishKafka processors in NIFI. Do these processors go to idle state if no data in incoming ?? if yes then what is the time in which they will go to idle state in case of no data is coming ??? and is there any way to set this time ourself ???
... View more
Labels:
- Labels:
-
Apache NiFi
12-12-2023
11:06 PM
disconnect and reconnect again this functionality i want to achieve with Publish kafka processor . ConsumerKafka will consumer continuously .
... View more
12-12-2023
11:04 PM
consuming from nifi is fine that i want to be continuous , i want to terminate connection for publish kafka processor , when there is no new message consumed by consumeKafka processor for more then 5 minutes , i want to terminate connection with publish kafka as my consumer kafka and producer kafka are different . as soon as my consumer kafka consumer some message , i want connection to be established again with producer kafka
... View more