Support Questions
Find answers, ask questions, and share your expertise

How to change cloudera navigator update interval?

New Contributor

We're testing Cloudera Navigator in CDH-5.4.x and would like to know if we can adjust the interval at which the navigator calculates the data lineage ?

It takes a long time before the newly created files (Hive tables, ...) to be available for querying through the web UI.

1 REPLY 1

Re: How to change cloudera navigator update interval?

Cloudera Employee

Hi, depending on the type of transformation (e.g., MR, HDFS, Hive, Impala, etc.), it can take anywhere from 10 minutes to ~1 hr for the full lineage and metadata to show up. We've done this by design, to ensure that Navigator metadata and lineage collection is performed in such a way that analysis is in no way impacted by Navigator's extractors that collect this information.

 

We are investigating ways to reduce this time while preserving Navigator's solid performance and scalability, but for now, we don't provide any publicly-supported mechanisms to increase the frequency of the extractor invocation.

 

Hope this helps.

 

Regards,

Mark.