- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Morphlines and multithreading
Created 10-16-2014 03:30 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi...
Does the single-threaded design of morphilnes mean that there can be concurrency issues by creating multiple morphlines from the same config file and running each of them in a separate thread?
For example, if one compiles 10 morphlines from the same config file and then run each of those in a separate thread, can that cause any thread safety issues?
Thanks.
Created 10-17-2014 12:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There?s no thread safety issue when doing that. (And there?s minimal contention as well, which improves throughput). In fact, that?s exactly what the hbase-indexer does in order to take advantage of multiple CPU cores, and that?s also what users of the Flume Morphline Solr Sink sometimes do.
Wolfgang.
Created 10-17-2014 12:02 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There?s no thread safety issue when doing that. (And there?s minimal contention as well, which improves throughput). In fact, that?s exactly what the hbase-indexer does in order to take advantage of multiple CPU cores, and that?s also what users of the Flume Morphline Solr Sink sometimes do.
Wolfgang.
Created 10-17-2014 01:04 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I assumed so, but wanted to confirm that there is no state sharing "behind the scenes", especially in the Command building process.
Thanks for the confirmation.
