Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
HDP 2.3

Index Life Cycle(Client events are in orange)-

8062-screen-shot-2016-09-28-at-35743-pm.png

Columns Names in SYSTEM.CATALOG table:-

Query:-

“select TABLE_NAME,DATA_TABLE_NAME,INDEX_TYPE,INDEX_STATE,INDEX_DISABLE_TIMESTAMP from system.catalog where INDEX_TYPE is not null;”

TABLE_NAME:-

Name of the index

DATA_TABLE_NAME:-

Name of the parent data table of index

INDEX_TYPE:-

GLOBAL(1)

LOCAL(2)

INDEX_STATE:-

Index States (Abbreviation stored in SYSTEM.CATALOG, states in bold are available to client too):-

BUILDING("b")

USABLE("e")

UNUSABLE("d")

ACTIVE("a")

INACTIVE("i")

DISABLE("x")

REBUILD("r")

(Below paragraph is from phoenix site)

DISABLE will cause the no further index maintenance to be performed on the index and it will no longer be considered for use in queries.

REBUILD will completely rebuild the index and upon completion will enable the index to be used in queries again.

BUILDING will partially rebuild the index from the last disabled timestamp and upon completion.

INACTIVE/UNUSABLE will cause the index to no longer be considered for use in queries, however index maintenance will continue to be performed.

ACTIVE/USABLE will cause the index to again be considered for use in queries.

Note that a disabled index must be rebuild and cannot be set as USABLE

INDEX_DISABLE_TIMESTAMP:-

It is the timestamp at which index is disabled. It will be 0 , if the index is active or disabled by client manually and will non-zero when index is disabled during write failures.

Automatic Rebuild process:-

  • MetaDataRegionObserver is responsible of running rebuild thread, so upsert select query to update the disabled index is executed from the regionserver , which is hosting SYSTEM.CATALOG table.
  • INACTIVE and DISABLE indexes are chosen for rebuild( provided all the regions of index table are online)
  • All Indexes of all tables are built serially.
  • We build the index from disabled timestamp - phoenix.index.failure.handling.rebuild.overlap.time(default 5 minutes) to SCN
  • Upsert select query used is "UPSERT /*+ NO_INDEX */ INTO index_table_name(indexedCols) select dataCols from data_table”

Index re-build lifecycle:-

Text Box: Rebuild successfulText Box: Rebuild startedRounded Rectangle: ActiveRounded Rectangle: InActiveRounded Rectangle: Disabled

  • 8063-screen-shot-2016-09-28-at-35855-pm.png

Text Box: Rebuild failed

Properties to control automatic Rebuilding :-

"phoenix.index.failure.handling.rebuild"(default true)

"phoenix.index.failure.handling.rebuild.interval" (default 10 seconds)

Consistency/Write Failures

https://phoenix.apache.org/secondary_indexing.html#Mutable Tables

2,774 Views
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 09:32 AM
Updated by:
 
Contributors
Top Kudoed Authors