Note in PutSQL I am using simple sql (insert into users_table(user_name) values ('user123'); It works properly for 10, 20 records but when I provide 100+ records then it gets stuck.
- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Flowfile is currently penalized in Apache NiFI between ExtractText and PutSQL processors in NiFi
- Labels:
-
Apache NiFi
Created 08-19-2024 04:37 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 08-22-2024 12:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello I had asked this question. Now I think I have got a solution. In PutSql Processor, there is a property "Support Fragmented Transactions". If I set its value to be false, then it is proceeding and not penalising the flow files.(If we read more into it, it says something about fragment.identifier and count due to which it waits for all flow files.)
Note: It also mentions it is for respecting Atomicity.
Anyone can suggest if having this property as false is proper or not. As right now if I am updating it to false then it is not penalising.
Created 08-19-2024 11:23 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@AlokKumar, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @MattWho @vaishaakb who may be able to assist you further.
Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.
Regards,
Vidya Sargur,Community Manager
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:
Created 08-22-2024 12:42 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks @VidyaSargur
Also I have replied which seems to be solution (property Support Framgmented Transactions as false in PutSql Processor) which seems to be helping.
Created 08-22-2024 12:41 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello I had asked this question. Now I think I have got a solution. In PutSql Processor, there is a property "Support Fragmented Transactions". If I set its value to be false, then it is proceeding and not penalising the flow files.(If we read more into it, it says something about fragment.identifier and count due to which it waits for all flow files.)
Note: It also mentions it is for respecting Atomicity.
Anyone can suggest if having this property as false is proper or not. As right now if I am updating it to false then it is not penalising.
