Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Pushing down Phoenix quey

avatar

I am trying to run a phoenix query, which scans millions of rows and does some processing. The problem is the entire data is being pulled to the client and the client runs out of memory. Is there a way to push down the query to the server side. IS it a setting that needs to be done for achieving this.

1 ACCEPTED SOLUTION

avatar
Master Guru

Depends on your processing. Can you send the query. Phoenix DOES push down local aggregations in the server side but does the full aggregation in the client. ( Like a Combiner ( Regionserver ) -> Reducer ( Client ) pattern.) One way to fix this can be salting of the keys to make sure specific keys are only in specific region servers.

For example if you have 100 regions and 100 group by keys, he will have to copy 100x100 rows to the client and merge them there. However if you make sure that each key is only present in one Region through salting he would only have to pull up 100x1 key.

But your query would help.

View solution in original post

1 REPLY 1

avatar
Master Guru

Depends on your processing. Can you send the query. Phoenix DOES push down local aggregations in the server side but does the full aggregation in the client. ( Like a Combiner ( Regionserver ) -> Reducer ( Client ) pattern.) One way to fix this can be salting of the keys to make sure specific keys are only in specific region servers.

For example if you have 100 regions and 100 group by keys, he will have to copy 100x100 rows to the client and merge them there. However if you make sure that each key is only present in one Region through salting he would only have to pull up 100x1 key.

But your query would help.