03-27-2018 12:58 AM - edited 03-27-2018 02:59 AM
I am using File tail in streamsets which would be pushing data to hdfs and other event to trash. After completion of all the process it is unable to delete from the directory.
The permission of the file in the directory is rw-rw-r-- 1 amank amank 24K Mar 27 12:38 finalOutput_2018-01-02.csv
The permission of the directoey is drwxrwxrwx 2 amank amank 4.0K Mar 27 12:38 data
2018-03-27 12:39:39,698 testFileTail
/testFTPSOURCE1138fa6c-18d0-4c58-84c8-85a267c7c28f INFO Writer
was idle closed, renaming.. RecordWriterManager *amank 0 Idle Close Thread
I belive there is bug in streamsets when I tried to push data from simple directory to hdfs . After completeion the file got deleted but not with file tail.