Created on 11-29-2022 04:22 AM - edited 11-29-2022 04:23 AM
I want to collect the metadata from HDFS. But when i searched , it looks like there is no atlas hook for HDFS available like it is there for kafka , hive and other data sources. Can I get the HDFS hook install steps or the steps to create a custom hook.
Created 11-29-2022 11:56 PM
Hello @Nigal ,
Created on 11-30-2022 05:45 AM - edited 11-30-2022 05:45 AM
Thanks for the solution . But this point i did'nt get clearly "hdfs_path is synced only if this belongs to a Hive table's lineage" . What i understood from this is that since hive runs on top of HDFS and on creating hive lineage, the lineage will show the HDFS path of hive warehouse directory. Is that correct?
Created 12-01-2022 10:20 PM