I want to collect the metadata from HDFS. But when i searched , it looks like there is no atlas hook for HDFS available like it is there for kafka , hive and other data sources. Can I get the HDFS hook install steps or the steps to create a custom hook.
Hello @Nigal ,
Thanks for the solution . But this point i did'nt get clearly "hdfs_path is synced only if this belongs to a Hive table's lineage" . What i understood from this is that since hive runs on top of HDFS and on creating hive lineage, the lineage will show the HDFS path of hive warehouse directory. Is that correct?