I have a program which runs every minute and uses STRACE command to make multiple log files simultaneously.
For example In first run - It might start tracing 10 processes with a log file name as unique PIDS, this will keep running till the entire stracing completes. In mean time second run would start, where it might start tracing few more processes ID's.
Now i want to continuosly stream all these STRACE logs to another server using flume.
I tried using spool but, since i have dynamic multiple real time files so, this was a bad option.
I thougt of using exec (tail) but as described the files names are multiple and quite dynamic so, could not figure out the best way to configure source.
Apprecite suggestions. Thanks.
Have you considered piping the STRACE output through syslogd, and using the Flume syslog source to get the events flowing through your Flume topology in realtime?