Reply
Highlighted
Explorer
Posts: 16
Registered: ‎11-27-2017

Unable to delete after processing in streamsets.

[ Edited ]

I am using File tail in streamsets which would be pushing data to hdfs and other event to trash. After  completion of all the process it is unable to delete from the directory.

 

The permission of the file in the directory is rw-rw-r-- 1 amank amank 24K Mar 27 12:38 finalOutput_2018-01-02.csv

 

The permission of the directoey is drwxrwxrwx 2 amank amank 4.0K Mar 27 12:38 data

 

Log:-

2018-03-27 12:39:39,698 testFileTail
/testFTPSOURCE1138fa6c-18d0-4c58-84c8-85a267c7c28f INFO Writer
for /user/amank/newData/testSms_tmp_sdc-7a4b66aa-22ca-11e8-9bee-a52e17517582_0.done
was idle closed, renaming.. RecordWriterManager *amank 0 Idle Close Thread

 I belive there is bug  in streamsets when I tried to push data from simple directory to hdfs . After completeion the file got deleted but not with file tail. 

Announcements