question Re: What does UNLINK event mean in hdfs inotify api? in Archives of Support Questions (Read Only)

What does UNLINK event mean in hdfs inotify api?

xiaqinglinv5 — Wed, 29 Mar 2017 09:49:45 GMT

Hi guys, I am working on writing a hdfs hook, yet I am not quite understand what does unlink event do, I found on this presentation that the delete log would be captured as a unlink event, but when I run a

hadoop fs -rm /h.txt

command, the unlink event is not captured? So what kind of hdfs commands would capture this unlink event ?

Re: What does UNLINK event mean in hdfs inotify api?

jsensharma — Wed, 29 Mar 2017 09:56:28 GMT

@Qinglin Xia

You might be interested in the Nifi GetHDFSEvents processor.

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.hadoop.inotify.GetHDFSEvents/

I personally did not check the UNLINK event though but i see a good example to validate the same. I try it you also might want to check the same in parallel:

https://github.com/onefoursix/hdfs-inotify-example/blob/master/src/main/java/com/onefoursix/HdfsINotifyExample.java

Re: What does UNLINK event mean in hdfs inotify api?

xiaqinglinv5 — Wed, 29 Mar 2017 13:06:06 GMT

Thanks I found those examples and run them already, but still not quite understand what unlink event do, also how the inotify works? Do you know any documentations explaining the inotify api?

Re: What does UNLINK event mean in hdfs inotify api?

xiaqinglinv5 — Wed, 29 Mar 2017 13:19:21 GMT

Finally got it, the delete op in hdfs should correspond to the rename event, coz hdfs would move the removed files/dirs to the trash file first