Hi there,   We don't currently capture this audit event, but I had opened an enhancement request internally to try to address this in a future release.   Mark. ... View more

Hi Rohan, We have plans to support cloud-based block storage for metadata discovery in C6, early next year. For now we will support metadata in block storage only in long running clusters through HDFS. So please stay tuned; this is definitely in our roadmap. Mark. ... View more

Hi Pradeep, Sounds like you already examined the Navigator logs and didn't see anything suspicious, right? Assuming that's the case, please either: 1 - provide your CM version number, Navigator version number, browser version, etc. OR: 2 - wait a few weeks when we ship C5.7. In C5.7 we have made a lot of optimizations to lineage when you try to render particularly complex diagrams. You would simply need to upgrade to CM5.7, which will also give you the option to upgrade to Navigator 2.6. You won't need to upgrade your CDH. Please let us know how things turn out and sorry for the trouble you've experienced. Mark. ... View more

Hi, I will follow up with the Optimizer team to see what I can find out.   Mark. ... View more

Hi there, you can request access to Navigator Optimizer by going here: https://optimizer.cloudera.com/ ... View more

Hi, to query for all entities that have tags, you can do the following query:   query=tags:*   As Vikas pointed out, to do a full extraction, you can do the following. Note that you'll need to do this in batches, using the limit and offset parameters. query=*   Also note that in the C5.5 release, we're introducing the Navigator SDK, which will make it easier to work with Navigator's metadata in Java. You may want to check out our Incremental Extraction example.   Regards, Mark. ... View more

Hi Elena,   Navigator has a number of "extractors" the connect to each compute engine and data store, such as Hive, Impala, HDFS, etc.  These extractors run several times an hour. The net result is that you have to wait about 20-30 minutes before full lineage appears in the UI. Navigator is zero-config for lineage so aside from the brief wait, there's nothing else you need to do for column-level lineage to automatically be collected. If it still doesn't show up, here are a few other tips:   - make sure you're running a supported version of CDH - the supported versions are listed in the documentation - check the Navigator log files from CM if all else fails - you might see some clues there, too    Mark  ... View more

Hi Venkat, Are you accessing auditing from the "Audits" tab of CM at port 7180? Or are you accessing Navigator from port 7187? As of a few releases ago, we moved Navigator's unified audit log out of CM and over to the Navigator user interface. You should be able to see all audit activity for HBase, Hive, HDFS and others by going to port 7187. ... View more

Hi, depending on the type of transformation (e.g., MR, HDFS, Hive, Impala, etc.), it can take anywhere from 10 minutes to ~1 hr for the full lineage and metadata to show up. We've done this by design, to ensure that Navigator metadata and lineage collection is performed in such a way that analysis is in no way impacted by Navigator's extractors that collect this information.   We are investigating ways to reduce this time while preserving Navigator's solid performance and scalability, but for now, we don't provide any publicly-supported mechanisms to increase the frequency of the extractor invocation.   Hope this helps.   Regards, Mark.   ... View more

Hi Hardeep,   Sorry for the delayed reply! We will be introducing a new Cloudera Navigator community forum soon so these messages don't get lost.   We introduced full support for Impala lineage in Cloudera Navigator 2.3, which shipped with Cloudera Enterprise 5.4 a few months ago. This includes column-level lineage for any transformations that are done in Impala, too.   Hope this helps & again, my apologies for not responding earlier.   Regards, Mark. ... View more