Reply
New Contributor
Posts: 5
Registered: ‎10-05-2016

Column level metadata

[ Edited ]

 

Hi All 

 

im looking for a little guidance, i have a requirement to load column/field level descriptions of CSV files so they are searchable. As a bit of context we will usually create external hive tables on top of these files for use by our analysts.

Is there a way to load this business metadata into navigator? we have full descriptions available from a copybook or metadata sheet, i was assuming i could write some service to call the navigator API to set these values for each column but i cannot see any documentation on how i might be able to do this.

Can someone help?

 

 

Cloudera Employee
Posts: 23
Registered: ‎08-13-2014

Re: Column level metadata

Hi,

Take a look at the API tutorial, it shows how to query entities in Navigator and how to apply metadata to them.

 

http://cloudera.github.io/navigator/apidocs/v3/tutorial.html

 

Start by crafting a query that selects the column you are interested. The Cloudera Navigator web UI is useful for this, make sure to click on show full query for a hint as to the query you can pass to the API. If for example you wanted to get the column 'orders' from the table 'c_customers' in database 'default':

 

+originalName:orders +parentPath:"/default/c_customers" +type:Field +sourceType:Hive +deleted:False

 

Using +deleted:False in your query ensures you get the current entity and not a previously deleted one. Once your familiar with finding the column you want to tag you can make a PUT request to upload the desired metadata. How to do this is covered in the documentation link I included above.

 

Regards,

Jim

Announcements
New Solutions