Issue: The lineage tab in the UI disappears when creating my own DataSet and Process types.
Details: Because I don’t want to inherit all the relationship attributes from DataSet, I would like to create my own custom, My_DataSet that inherits directly from Asset, just the same the Atlas’ buil-in one.
Also, because of the relationships defined between DataSet and Process, I also created my own process type, say My_Process and therefore linked the new types through the new,
my_dataset_process_inputs and
my_process_dataset_outputs.
These relationships, just like the buil-in Atlas’ ones tie together the relationships attributes My_DataSet.inputToProcesses, My_DataSet.inputFromProcesses to My_Process.inputs and My_Process.outputs. So just like Atlas, the only difference, less relationship attributes in the My_DataSet definition, as mentioned before.
Back to the issue, now I don’t see the “Lineage” tab in the UI. The relationships between actual entities, when created, are shown properly, but lineage capability, appears to be gone.
Questions:
1. I wonder if Atlas relies on its own, dataset_process_inputs and
process_dataset_outputs, with these exact names to infer the concept of lineage?
2. If the above is true, I would think that somehow I should be able to tell Atlas that my own, my_dataset_process_inputs and my_process_dataset_outputs should also be used to make lineage available?
If this won’t be possible, it would be very constraining. Should be able to create a custom model, one would think. Hopefully it is not the case.
I would appreciate any education/advise you could provide.
Thanks in advance!
