Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

What does UNLINK event mean in hdfs inotify api?

avatar
Contributor

Hi guys, I am working on writing a hdfs hook, yet I am not quite understand what does unlink event do, I found on this presentation that the delete log would be captured as a unlink event, but when I run a

hadoop fs -rm /h.txt 

command, the unlink event is not captured? So what kind of hdfs commands would capture this unlink event ?

1 ACCEPTED SOLUTION

avatar
Master Mentor

@Qinglin Xia

You might be interested in the Nifi GetHDFSEvents processor.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.inotify.GetHDFSE...

.

I personally did not check the UNLINK event though but i see a good example to validate the same. I try it you also might want to check the same in parallel:

https://github.com/onefoursix/hdfs-inotify-example/blob/master/src/main/java/com/onefoursix/HdfsINot...

.

View solution in original post

3 REPLIES 3

avatar
Master Mentor

@Qinglin Xia

You might be interested in the Nifi GetHDFSEvents processor.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.inotify.GetHDFSE...

.

I personally did not check the UNLINK event though but i see a good example to validate the same. I try it you also might want to check the same in parallel:

https://github.com/onefoursix/hdfs-inotify-example/blob/master/src/main/java/com/onefoursix/HdfsINot...

.

avatar
Contributor

Thanks I found those examples and run them already, but still not quite understand what unlink event do, also how the inotify works? Do you know any documentations explaining the inotify api?

avatar
Contributor

Finally got it, the delete op in hdfs should correspond to the rename event, coz hdfs would move the removed files/dirs to the trash file first