question Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types in Support Questions

The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Wed, 16 Oct 2024 17:05:43 GMT

Issue: The lineage tab in the UI disappears when creating my own DataSet and Process types.

Details: Because I don’t want to inherit all the relationship attributes from DataSet, I would like to create my own custom, My_DataSet that inherits directly from Asset, just the same the Atlas’ buil-in one.

Also, because of the relationships defined between DataSet and Process, I also created my own process type, say My_Process and therefore linked the new types through the new,

my_dataset_process_inputs and

my_process_dataset_outputs.

These relationships, just like the buil-in Atlas’ ones tie together the relationships attributes My_DataSet.inputToProcesses, My_DataSet.inputFromProcesses to My_Process.inputs and My_Process.outputs. So just like Atlas, the only difference, less relationship attributes in the My_DataSet definition, as mentioned before.

Back to the issue, now I don’t see the “Lineage” tab in the UI. The relationships between actual entities, when created, are shown properly, but lineage capability, appears to be gone.

Questions:

1. I wonder if Atlas relies on its own, dataset_process_inputs and

process_dataset_outputs, with these exact names to infer the concept of lineage?

2. If the above is true, I would think that somehow I should be able to tell Atlas that my own, my_dataset_process_inputs and my_process_dataset_outputs should also be used to make lineage available?

If this won’t be possible, it would be very constraining. Should be able to create a custom model, one would think. Hopefully it is not the case.

I would appreciate any education/advise you could provide.

Thanks in advance!

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

DianaTorres — Wed, 16 Oct 2024 19:26:37 GMT

@Heli Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Atlas experts @Scharan @cravani who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Wed, 16 Oct 2024 20:16:19 GMT

Thank you Diana, really appreciated!

New to the community and looking forward to learn.

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Thu, 17 Oct 2024 15:13:45 GMT

Hi @Scharan and @cravani,

I would very much appreciate if you could assist please?

Thank you.

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Tue, 22 Oct 2024 14:11:33 GMT

Hi, does anybody know the answer to my question? I understand it may not be a trivial question, but I thought the experts here may have an idea.

@DianaTorres, would you know of anybody else who would be able to help?

If there is a more appropriate forum/community, I should post this question, please share it.

Thanks.

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

DianaTorres — Tue, 22 Oct 2024 17:47:18 GMT

Hi! @pvishnu @mayank_gupta Do you have any insights here? Thanks!

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Fri, 25 Oct 2024 16:31:32 GMT

Hi @pvishnu, @mayank_gupta

I would really appreciate if you could help.

Thanks in advance.

@DianaTorres, thank you for trying, it is a little disappointing nobody bothers to give this a try. Oh well, it is what it is! thanks again Diana. At this point I won't get my hopes up.

Re: The "Lineage" tab in the UI disappears when creating my own DataSet and Process types

Heli — Fri, 25 Oct 2024 16:34:32 GMT

Forgot to mention, is there something I said that rendered no responses? My apologies if that's the case, didn't think I ofended anybody.