Support Questions

Find answers, ask questions, and share your expertise

Is hdfs metadata type already defined in atlas 0.7?

avatar
Contributor

I noticed atlas has put hdfs-path type in the 0.7 version, what does hdfs-path do in hdfs metadata, and if I want to create a hdfs bridge/hook myself, do I need to create additional types in atlas?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

hdfs_path refers to any hdfs location (file,directory,symlink etc.,). Since hdfs_path is already registered in Atlas, you can write a hook in hdfs to capture metadata change events and construct the json as per hdfs_path type definition and POST it to Atlas.

Also , you can create your own custom type and register it.

Since you are writing own hook libraries for hdfs , your hook library can construct the json to use the existing type system in 0.7 or your custom type you registered.

View solution in original post

1 REPLY 1

avatar
Super Collaborator

hdfs_path refers to any hdfs location (file,directory,symlink etc.,). Since hdfs_path is already registered in Atlas, you can write a hook in hdfs to capture metadata change events and construct the json as per hdfs_path type definition and POST it to Atlas.

Also , you can create your own custom type and register it.

Since you are writing own hook libraries for hdfs , your hook library can construct the json to use the existing type system in 0.7 or your custom type you registered.