Pushing down Phoenix quey

arunpoy — Wed, 29 Jun 2016 18:29:12 GMT

I am trying to run a phoenix query, which scans millions of rows and does some processing. The problem is the entire data is being pulled to the client and the client runs out of memory. Is there a way to push down the query to the server side. IS it a setting that needs to be done for achieving this.

Re: Pushing down Phoenix quey

bleonhardi — Wed, 29 Jun 2016 19:13:18 GMT

Depends on your processing. Can you send the query. Phoenix DOES push down local aggregations in the server side but does the full aggregation in the client. ( Like a Combiner ( Regionserver ) -> Reducer ( Client ) pattern.) One way to fix this can be salting of the keys to make sure specific keys are only in specific region servers.

For example if you have 100 regions and 100 group by keys, he will have to copy 100x100 rows to the client and merge them there. However if you make sure that each key is only present in one Region through salting he would only have to pull up 100x1 key.

But your query would help.

question Re: Pushing down Phoenix quey in Archives of Support Questions (Read Only)

Pushing down Phoenix quey

Re: Pushing down Phoenix quey