Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Is there a limitation on a number of secondary indexes I can define on Phoenix table?

avatar
Master Mentor

If I have a requirement to index 10 columns individually, would that be an appropriate use of Phoenix secondary indexes. Is there a huge impact with a large number of indexes to maintain, what is the threshold? I understand it's a loaded question but let's pretend cluster size is 16 nodes, 100GB daily ingest rate.

1 ACCEPTED SOLUTION

avatar
Super Guru

(assuming you're referring to mutable global indexes..)

There is a direct relation to the amount of index data you have to create for every update your write to the data table. So, if you are indexing 10 columns for a data table, you're actually writing 11 updates for every one 1 update your clients writes.

Only 100GB of data for a day on 16nodes seems like it would be reasonable to manage this, but you are pushing a lot of work to the RegionServers. I would make sure that the RegionServers are adequately sized to handle all of the extra load.

Using immutable tables pushes this work to the client which might be more scalable a solution: https://phoenix.apache.org/secondary_indexing.html#Immutable_Tables

View solution in original post

2 REPLIES 2

avatar
Super Guru

(assuming you're referring to mutable global indexes..)

There is a direct relation to the amount of index data you have to create for every update your write to the data table. So, if you are indexing 10 columns for a data table, you're actually writing 11 updates for every one 1 update your clients writes.

Only 100GB of data for a day on 16nodes seems like it would be reasonable to manage this, but you are pushing a lot of work to the RegionServers. I would make sure that the RegionServers are adequately sized to handle all of the extra load.

Using immutable tables pushes this work to the client which might be more scalable a solution: https://phoenix.apache.org/secondary_indexing.html#Immutable_Tables

avatar
Master Guru

Josh that link you shared is priceless.