HDFS_PATH in Apache Atlas Search

[#1]

I am trying to understand the exact significance of hdfs_path in Apache Atlas Search. I created a path in HDFS but i am not able to search that path using Atlas UI. Does it signify something else?

I have installed HDP 2.5 and using Atlas on HDP 2.5. One more thing Is Atlas Web UI browser specific? I am able to search newly created hive tables in IE but not in Chrome and Firefox.?

[#2 · Accepted solution]

Hi @rahul gulati,

Atlas GUI works for Chrome and Firefox. I'm using HDP 2.5.3 and the latest Chrome version without any problems.

hdfs_path is an Atlas metadata type but unfortunately it's no HDFS hook available in 2.5 which automatically populates the hdfs_path so you probably do not have any hdfs entity data in Atlas. HDP 2.6 will include some functionality for HDFS entity creation.

You can always build your own hook - a good doc to understand the Atlas model is http://atlas.incubator.apache.org/0.7.1-incubating/AtlasTechnicalUserGuide.pdf.

/Best regards, Mats

[#3]

@Mats Johansson

Thanks for sharing the technical User guide.

Which version of Google Chrome are you using. I am unable to search newly created Hive tables from Atlas UI Text search?

[#4]

Hi,

The easiest way to find your entity:

1/- On Basic/advanced mode:

- select hive_db on Search by Type

2/- Or on advanced mode (DSL), you could filter on Search by Query & just put that: hive_db.

To filter by owner, for instance:

hive_db where owner="public"

[#5]

Using HDP2.6 with Atlas 0.8, I observe the same behaviour: "hdfs_path based requests does not produce any results... (no records found)"

At this moment, does it exist a working HDFS hook for Atlas?

[#6]

You should create a Hive external table with that HDFS path then it will show up

[#7]

Is there some example to create Hive table to show up files of the hdfs_path entity? Thank you.