Member since
07-12-2017
53
Posts
3
Kudos Received
0
Solutions
07-18-2024
10:32 PM
431 / 5,000 Resultados de traducción Traducción And what is done in this case: Nothing about how the content repository works will have any impact on NiFi UI performance. A full content repository will howvere impact the overall performance of your NiFi dataflows. Because when you change nifi.content.repository.archive.enabled to false, in a cluster of 3 nodes, for example, nifi only raises 2 nodes, always the first one that starts and the second, that remains visible in the cluster, but the third , is left running only as a 1-1 clusterer
... View more
07-21-2021
11:27 PM
@Kasun, as this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.
... View more
10-22-2020
04:18 PM
@mburgess Helpful information shared. I am using Nifi 1.7.1 For my case, the incremental fetching does not seem to work correctly. All records gets ingested from the database but do not make it all the way to the destination. Processor used is GenerateTableFetch then Execute SQL and the other corresponding processors down the data processing flow. Record id is captured correctly on the GenerateTableFetch property state and its up to date as the record id from the source (db). However, it will still miss some records when processing the files making the number of records on the destination out of sync with the source from the db. Am i missing something, Would scheduling times for fetching help and how can I do that?
... View more
04-03-2019
01:59 PM
Thanks Matt for your view on this,the ask is to generate a batchid which should be a sequence number, so whenever querydb processor fetches records from source db (sqlserver) a batchid should be added to the flowfile so that all records have a same batchid when loaded to target table,this will help in auditing of records.but here in cluster mode it seems difficult to achieve this using updateattribute processor.i liked your idea of appending node hostname with the sequence but if i could generate atomic values across all nodes it would be much better.
... View more
03-20-2019
02:02 PM
@Matt Clarke,@matt burgess Exactly the second point is happening, each node is generating its own value incrementing from last value it has stored in its local state. So which processor or method should i use to generate an incremental batchid (batch1,batch2...so on) since update attribute is messing values when running on cluster. or is there any property by which updateattribute processors on all nodes can pickup each others's last state variable?..please suggest
... View more
09-14-2018
01:47 PM
@sri chaturvedi yes, potentially if there are enough inbound FlowFiles to trigger processor to run 4 times concurrently.
... View more
04-11-2018
05:46 AM
Thanks for the solution, but since i am not familiar with rest api, solution by Matt looks easy to me. Will surely try yours one too.
... View more
04-09-2018
07:51 PM
@sri chaturvedi I have captured above in a new article: https://community.hortonworks.com/articles/184786/hdfnifi-improving-the-performance-of-your-ui.html
... View more
03-20-2019
06:45 AM
@Shu will this work even if we have some state alraedy stored withing the processor? for eg: i have a timestamp (2019-03-17 02:00:00:0) stored in the state of my processor now i want the processor to start fetching data after 2019-03-20, will this property help in such scenario?
... View more
11-09-2017
11:17 AM
@Abdelkrim Hadjidj Thanks a lot for your inputs into this, i will try this, i think this will surely solve my issue.
... View more