Support Questions

Find answers, ask questions, and share your expertise

HDFS_PATH in Apache Atlas Search

avatar
Rising Star

I am trying to understand the exact significance of hdfs_path in Apache Atlas Search. I created a path in HDFS but i am not able to search that path using Atlas UI. Does it signify something else?

I have installed HDP 2.5 and using Atlas on HDP 2.5. One more thing Is Atlas Web UI browser specific? I am able to search newly created hive tables in IE but not in Chrome and Firefox.?

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hi @rahul gulati,

Atlas GUI works for Chrome and Firefox. I'm using HDP 2.5.3 and the latest Chrome version without any problems.

hdfs_path is an Atlas metadata type but unfortunately it's no HDFS hook available in 2.5 which automatically populates the hdfs_path so you probably do not have any hdfs entity data in Atlas. HDP 2.6 will include some functionality for HDFS entity creation.

You can always build your own hook - a good doc to understand the Atlas model is http://atlas.incubator.apache.org/0.7.1-incubating/AtlasTechnicalUserGuide.pdf.

/Best regards, Mats

View solution in original post

6 REPLIES 6

avatar
Super Collaborator

Hi @rahul gulati,

Atlas GUI works for Chrome and Firefox. I'm using HDP 2.5.3 and the latest Chrome version without any problems.

hdfs_path is an Atlas metadata type but unfortunately it's no HDFS hook available in 2.5 which automatically populates the hdfs_path so you probably do not have any hdfs entity data in Atlas. HDP 2.6 will include some functionality for HDFS entity creation.

You can always build your own hook - a good doc to understand the Atlas model is http://atlas.incubator.apache.org/0.7.1-incubating/AtlasTechnicalUserGuide.pdf.

/Best regards, Mats

avatar
Rising Star

@Mats Johansson

Thanks for sharing the technical User guide.

Which version of Google Chrome are you using. I am unable to search newly created Hive tables from Atlas UI Text search?

avatar
Contributor

Hi,

The easiest way to find your entity:

1/- On Basic/advanced mode:

- select hive_db on Search by Type

2/- Or on advanced mode (DSL), you could filter on Search by Query & just put that: hive_db.

To filter by owner, for instance:

hive_db where owner="public"

avatar
New Contributor

Using HDP2.6 with Atlas 0.8, I observe the same behaviour: "hdfs_path based requests does not produce any results... (no records found)"

At this moment, does it exist a working HDFS hook for Atlas?

avatar
New Contributor

You should create a Hive external table with that HDFS path then it will show up

avatar
Explorer

Is there some example to create Hive table to show up files of the hdfs_path entity? Thank you.