Reply
New Contributor
Posts: 5
Registered: ‎12-22-2017

oryx 2 can't update result for serving layer

[ Edited ]

I use oryx 2.6.
System environment:
cdh 5.13.0 + kafka 3.0.0+ spark 2.2.0

 

This is not the same as http://oryx.io/docs/admin.html:
hadloop is 2.6 not 2.7
spark-kafka only 0.10

 

my question:
1, als at the serving layer can not get the latest calculated data:
E.g:
I ingest the following data:
1000,22
1001,33

 

But I can not see this '1001' data

In addition, the speed layer can not update the data, it looks like there is calculation, but did not update the data to the serving layer

 

2, wordcount example project, serving layer can not view the data.
I find:
It has the data stored in hdfs, kafka's topic message is correct. But the batch layer updates are empty, such as: {}

 

Is my environmental problem? Or these are bug?

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: oryx 2 can't update result for serving layer

I'll answer in the other duplicated post.

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: oryx 2 can't update result for serving layer

It could be a lot of things, but keep in mind that you will not see any model updates unless you have a batch layer running, and it has had time to compute a model and send it to the serving layer. If the batch layer is running, check if it is able to produce a model. It would not with only 2 data points.

New Contributor
Posts: 5
Registered: ‎12-22-2017

Re: oryx 2 can't update result for serving layer

[ Edited ]

Thanks.

1、Is this wrong?

"Speed Layer, which produces and publishes incremental model updates from a stream of new data"

 

2、sorry, I did not say clearly on the question 1.

E.g:

First, I enter these data:
1000,22
1001,33
Later, after the batch level calculation, I can not see the data: 1001,33


Then I enter these data:
1002,44
Then I can see the data: 1001,33, but can not see the data: 1002,44

 

I will never see the latest data

 

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: oryx 2 can't update result for serving layer

No, because the speed layer also can't produce model updates unless it first sees a model.

 

What do you mean that you can't see the data -- in what place?

New Contributor
Posts: 5
Registered: ‎12-22-2017

Re: oryx 2 can't update result for serving layer

[ Edited ]

1, as you said, the speed of the layer can not be incremental updates, and the batch layer implementation takes some time,
How to solve real-time?
2, I check the data access to the serving layer: '/user/allIDs' and '/item/allIDs'

 

3、I found the answer to the second question: wordcount example project.

I only add one word at a time,Then It can not be counted.

If I add more than one word at a time,Then It is ok.

I think there are some problems in calculating part of the code in example,
I can solve this problem by myself.

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: oryx 2 can't update result for serving layer

You have to have an initial model before anything works. After that, of course, model scoring happens in real time and updates happen in near-real-time.

 

I'm not sure what you mean in your second point.

 

The word count example is correct. It's counting unique co-occurrences of words. If there is just one word on a line, there are no co-occurrences to count.

New Contributor
Posts: 5
Registered: ‎12-22-2017

Re: oryx 2 can't update result for serving layer

 

Based on my test.

Speed layer can not update data for serving layer, I can not get the last chaged data from serving layer.

why you said that 'It can updates happen in near-real-time'?

Is it wrong for me to understand?

 

Ignore the second point.

 

 

 

Cloudera Employee
Posts: 481
Registered: ‎08-11-2014

Re: oryx 2 can't update result for serving layer

I'm not sure what you're asking here. I have verified the project build works and all tests pass. Follow the tutorial at http://oryx.io/docs/endusers.html to get a working instance and take it from there.

New Contributor
Posts: 5
Registered: ‎12-22-2017

Re: oryx 2 can't update result for serving layer

Oh, I see. Thank you anyway.

Announcements