I am getting below error while running Flume agent to spool directory,
2016-06-16 10:07:35,927 INFO org.apache.flume.source.SpoolDirectorySource: SpoolDirectorySource source starting with directory: /export/home/user1/flume/input
2016-06-16 10:07:35,927 ERROR org.apache.flume.lifecycle.LifecycleSupervisor: Unable to start EventDrivenSourceRunner: { source:Spool Directory source sdir: { spoolDir: /export/home/user1/flume/input } } - Exception follows.
java.lang.IllegalStateException: Directory does not exist: /export/home/user1/flume/input
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.<init>(ReliableSpoolingFileEventReader.java:145)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.<init>(ReliableSpoolingFileEventReader.java:77)
at org.apache.flume.client.avro.ReliableSpoolingFileEventReader$Builder.build(ReliableSpoolingFileEventReader.java:669)
at org.apache.flume.source.SpoolDirectorySource.start(SpoolDirectorySource.java:85)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
- I am running Flume as service on Edge node.
- Directory '/export/home/user1/flume/input' is on same Edge node, but this direcory is owned by user1
- I assume Flume agent on Edge node run by 'Flume' as user
- May be error is because 'Flume' user not having r / w access on folder '/export/home/user1/flume/input'
If my understanding is correct then I must create a spool directory where Flume as user have r/w access and other users posting files in that folder have at-least write permission.
Please correct me if my understanding is not correct.
