Pushing down Phoenix quey

[#1]

I am trying to run a phoenix query, which scans millions of rows and does some processing. The problem is the entire data is being pulled to the client and the client runs out of memory. Is there a way to push down the query to the server side. IS it a setting that needs to be done for achieving this.

[#2 · Accepted solution]

Depends on your processing. Can you send the query. Phoenix DOES push down local aggregations in the server side but does the full aggregation in the client. ( Like a Combiner ( Regionserver ) -> Reducer ( Client ) pattern.) One way to fix this can be salting of the keys to make sure specific keys are only in specific region servers.

For example if you have 100 regions and 100 group by keys, he will have to copy 100x100 rows to the client and merge them there. However if you make sure that each key is only present in one Region through salting he would only have to pull up 100x1 key.

But your query would help.