Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oryx Class Diagram

Oryx Class Diagram

Explorer

Hello

 

I'm trying to find some document that shows the architecture of Oryx in full details. something like the Class diagram or...

in the main website there is only the explanation and some simple figure that does not show how exactly different components are connecting together ...

components are connecting together ...

 

any help ?

 

6 REPLIES 6

Re: Oryx Class Diagram

Master Collaborator

I am not sure that a class diagram would be so helpful, as it doesn't capture the structure of the project much. However you can look up some free tools that can generate UML from a Java project and those would work. I think the architectural overview here is most useful: http://oryxproject.github.io/oryx/ and module breakdown here: http://oryxproject.github.io/oryx/docs/index.html#Module_Mapping are what you want.

Highlighted

Re: Oryx Class Diagram

Explorer

Hello again dear Sowen

 

in following I write the sequence of what is happening during oryx , according to my understanding. could you please correct me ?

 

 

 

1- there is a HDFS directory which Batch layer has access to it. so it will go there , read the data and compute the model and then it rests 

untill next time next time . ( which is based on our configuration. we can set the interval based of our need) --> my question is where it

will store the Model? in the same HDFS directory? OR it send the model instantly to serving layer (using kafka) but anyway the model should

store somewhere that serving be able to use it...

 

2-then  Serving layer will find the model (somehow) , and can answer the queries from the user based on the model.

 

3- as time passes we may have new data , so this new data will be written again in the HDFS directory (exactly the same as previous ?? ), 

and the serving layer will send this data to the speed layer ( or speed layer directly read this data form HDFS ?) and speed layer will compute

the update model ...

 

4-now the serving layer will find this new model use it model to answer the queries

 

 

 

Re: Oryx Class Diagram

Master Collaborator

Well, the batch layer reads input from Kafka. It writes the input to HDFS, but also reads previous input from HDFS and uses all of it to build a model. The model is sent to a different Kafka topic. (It is also written to HDFS actually, in case other apps want to consume it that way.) Serving layer reads the Kafka topic to get the model. Serving layer can also write input to the input topic. No, nothing reads from HDFS except the batch layer.

Re: Oryx Class Diagram

Explorer
thank you so much, and so the speed layer does the same?..it reads input from kafka and then send the model to kafka ... is it right ?

Re: Oryx Class Diagram

Master Collaborator

It reads models, reads input, and writes model updates, not models.

Re: Oryx Class Diagram

Explorer
Thanks alot :)