Member since
01-28-2014
13
Posts
3
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3206 | 12-02-2016 03:57 PM |
12-12-2016
11:43 AM
I'm sorry - I don't understand the question. Can you please clarify?
... View more
12-02-2016
03:57 PM
1 Kudo
Hi, our documentation describes the behaviour of Oozie lineage at http://www.cloudera.com/documentation/enterprise/latest/topics/cn_iu_lineage.html. For specifics, have you tried running Oozie workflows and examining them inside Navigator? If you don't have a license yet, you can always get a trial license from cloudera.com to try this out for yourself. Hope this helps.
... View more
12-02-2016
03:52 PM
Hi there, Although this might work right now, we're using a different algorithm to calculate the entity ID starting in C5.10, which ships next month. We'll automatically translate from the old entity ID to the new one transparently, but you're best off using our API to determine the entity ID -- specifically: api/v10/entities/?query=<entity_query> I hope this helps. Regards, Mark.
... View more
07-14-2016
12:19 PM
Hi there, We don't currently capture this audit event, but I had opened an enhancement request internally to try to address this in a future release. Mark.
... View more
04-01-2016
08:32 PM
Hi Rohan, We have plans to support cloud-based block storage for metadata discovery in C6, early next year. For now we will support metadata in block storage only in long running clusters through HDFS. So please stay tuned; this is definitely in our roadmap. Mark.
... View more
04-01-2016
08:29 PM
1 Kudo
Hi Pradeep, Sounds like you already examined the Navigator logs and didn't see anything suspicious, right? Assuming that's the case, please either: 1 - provide your CM version number, Navigator version number, browser version, etc. OR: 2 - wait a few weeks when we ship C5.7. In C5.7 we have made a lot of optimizations to lineage when you try to render particularly complex diagrams. You would simply need to upgrade to CM5.7, which will also give you the option to upgrade to Navigator 2.6. You won't need to upgrade your CDH. Please let us know how things turn out and sorry for the trouble you've experienced. Mark.
... View more
01-05-2016
09:37 AM
Hi, I will follow up with the Optimizer team to see what I can find out. Mark.
... View more
12-09-2015
02:13 PM
Hi there, you can request access to Navigator Optimizer by going here: https://optimizer.cloudera.com/
... View more
11-16-2015
02:01 PM
Hi, to query for all entities that have tags, you can do the following query: query=tags:* As Vikas pointed out, to do a full extraction, you can do the following. Note that you'll need to do this in batches, using the limit and offset parameters. query=* Also note that in the C5.5 release, we're introducing the Navigator SDK, which will make it easier to work with Navigator's metadata in Java. You may want to check out our Incremental Extraction example. Regards, Mark.
... View more
08-27-2015
09:54 PM
Hi Elena, Navigator has a number of "extractors" the connect to each compute engine and data store, such as Hive, Impala, HDFS, etc. These extractors run several times an hour. The net result is that you have to wait about 20-30 minutes before full lineage appears in the UI. Navigator is zero-config for lineage so aside from the brief wait, there's nothing else you need to do for column-level lineage to automatically be collected. If it still doesn't show up, here are a few other tips: - make sure you're running a supported version of CDH - the supported versions are listed in the documentation - check the Navigator log files from CM if all else fails - you might see some clues there, too Mark
... View more
08-14-2015
02:58 PM
Hi Venkat, Are you accessing auditing from the "Audits" tab of CM at port 7180? Or are you accessing Navigator from port 7187? As of a few releases ago, we moved Navigator's unified audit log out of CM and over to the Navigator user interface. You should be able to see all audit activity for HBase, Hive, HDFS and others by going to port 7187.
... View more
08-03-2015
02:10 PM
1 Kudo
Hi, depending on the type of transformation (e.g., MR, HDFS, Hive, Impala, etc.), it can take anywhere from 10 minutes to ~1 hr for the full lineage and metadata to show up. We've done this by design, to ensure that Navigator metadata and lineage collection is performed in such a way that analysis is in no way impacted by Navigator's extractors that collect this information. We are investigating ways to reduce this time while preserving Navigator's solid performance and scalability, but for now, we don't provide any publicly-supported mechanisms to increase the frequency of the extractor invocation. Hope this helps. Regards, Mark.
... View more
08-03-2015
02:07 PM
Hi Hardeep, Sorry for the delayed reply! We will be introducing a new Cloudera Navigator community forum soon so these messages don't get lost. We introduced full support for Impala lineage in Cloudera Navigator 2.3, which shipped with Cloudera Enterprise 5.4 a few months ago. This includes column-level lineage for any transformations that are done in Impala, too. Hope this helps & again, my apologies for not responding earlier. Regards, Mark.
... View more