Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

oryx 2 can't update result for serving layer

oryx 2 can't update result for serving layer

New Contributor

I use oryx 2.6.
System environment:
cdh 5.13.0 + kafka 3.0.0+ spark 2.2.0

 

This is not the same as http://oryx.io/docs/admin.html:
hadloop is 2.6 not 2.7
spark-kafka only 0.10

 

my question:
1, als at the serving layer can not get the latest calculated data:
E.g:
I ingest the following data:
1000,22
1001,33

 

But I can not see this '1001' data

In addition, the speed layer can not update the data, it looks like there is calculation, but did not update the data to the serving layer

 

2, wordcount example project, serving layer can not view the data.
I find:
It has the data stored in hdfs, kafka's topic message is correct. But the batch layer updates are empty, such as: {}

 

Is my environmental problem? Or these are bug?

9 REPLIES 9

Re: oryx 2 can't update result for serving layer

Master Collaborator

I'll answer in the other duplicated post.

Re: oryx 2 can't update result for serving layer

Master Collaborator

It could be a lot of things, but keep in mind that you will not see any model updates unless you have a batch layer running, and it has had time to compute a model and send it to the serving layer. If the batch layer is running, check if it is able to produce a model. It would not with only 2 data points.

Highlighted

Re: oryx 2 can't update result for serving layer

New Contributor

Thanks.

1、Is this wrong?

"Speed Layer, which produces and publishes incremental model updates from a stream of new data"

 

2、sorry, I did not say clearly on the question 1.

E.g:

First, I enter these data:
1000,22
1001,33
Later, after the batch level calculation, I can not see the data: 1001,33


Then I enter these data:
1002,44
Then I can see the data: 1001,33, but can not see the data: 1002,44

 

I will never see the latest data

 

Re: oryx 2 can't update result for serving layer

Master Collaborator

No, because the speed layer also can't produce model updates unless it first sees a model.

 

What do you mean that you can't see the data -- in what place?

Re: oryx 2 can't update result for serving layer

New Contributor

1, as you said, the speed of the layer can not be incremental updates, and the batch layer implementation takes some time,
How to solve real-time?
2, I check the data access to the serving layer: '/user/allIDs' and '/item/allIDs'

 

3、I found the answer to the second question: wordcount example project.

I only add one word at a time,Then It can not be counted.

If I add more than one word at a time,Then It is ok.

I think there are some problems in calculating part of the code in example,
I can solve this problem by myself.

Re: oryx 2 can't update result for serving layer

Master Collaborator

You have to have an initial model before anything works. After that, of course, model scoring happens in real time and updates happen in near-real-time.

 

I'm not sure what you mean in your second point.

 

The word count example is correct. It's counting unique co-occurrences of words. If there is just one word on a line, there are no co-occurrences to count.

Re: oryx 2 can't update result for serving layer

New Contributor

 

Based on my test.

Speed layer can not update data for serving layer, I can not get the last chaged data from serving layer.

why you said that 'It can updates happen in near-real-time'?

Is it wrong for me to understand?

 

Ignore the second point.

 

 

 

Re: oryx 2 can't update result for serving layer

Master Collaborator

I'm not sure what you're asking here. I have verified the project build works and all tests pass. Follow the tutorial at http://oryx.io/docs/endusers.html to get a working instance and take it from there.

Re: oryx 2 can't update result for serving layer

New Contributor

Oh, I see. Thank you anyway.

Don't have an account?
Coming from Hortonworks? Activate your account here