Created 10-16-2024 10:05 AM
Issue: The lineage tab in the UI disappears when creating my own DataSet and Process types.
Details: Because I don’t want to inherit all the relationship attributes from DataSet, I would like to create my own custom, My_DataSet that inherits directly from Asset, just the same the Atlas’ buil-in one.
Also, because of the relationships defined between DataSet and Process, I also created my own process type, say My_Process and therefore linked the new types through the new,
my_dataset_process_inputs and
my_process_dataset_outputs.
These relationships, just like the buil-in Atlas’ ones tie together the relationships attributes My_DataSet.inputToProcesses, My_DataSet.inputFromProcesses to My_Process.inputs and My_Process.outputs. So just like Atlas, the only difference, less relationship attributes in the My_DataSet definition, as mentioned before.
Back to the issue, now I don’t see the “Lineage” tab in the UI. The relationships between actual entities, when created, are shown properly, but lineage capability, appears to be gone.
Questions:
1. I wonder if Atlas relies on its own, dataset_process_inputs and
process_dataset_outputs, with these exact names to infer the concept of lineage?
2. If the above is true, I would think that somehow I should be able to tell Atlas that my own, my_dataset_process_inputs and my_process_dataset_outputs should also be used to make lineage available?
If this won’t be possible, it would be very constraining. Should be able to create a custom model, one would think. Hopefully it is not the case.
I would appreciate any education/advise you could provide.
Thanks in advance!
Created 10-16-2024 12:26 PM
@Heli Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our Atlas experts @Scharan @cravani who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Created 10-16-2024 01:16 PM
Thank you Diana, really appreciated!
New to the community and looking forward to learn.
Created 10-17-2024 08:13 AM
Created 10-22-2024 07:11 AM
Hi, does anybody know the answer to my question? I understand it may not be a trivial question, but I thought the experts here may have an idea.
@DianaTorres, would you know of anybody else who would be able to help?
If there is a more appropriate forum/community, I should post this question, please share it.
Thanks.
Created 10-22-2024 10:47 AM
Hi! @pvishnu @mayank_gupta Do you have any insights here? Thanks!
Regards,
Diana Torres,Created 10-25-2024 09:31 AM
I would really appreciate if you could help.
Thanks in advance.
@DianaTorres, thank you for trying, it is a little disappointing nobody bothers to give this a try. Oh well, it is what it is! thanks again Diana. At this point I won't get my hopes up.
Created 10-25-2024 09:34 AM
Forgot to mention, is there something I said that rendered no responses? My apologies if that's the case, didn't think I ofended anybody.