Created on 09-28-201612:30 PM - edited 08-17-201909:32 AM
HDP
2.3
Index Life Cycle(Client events are
in orange)-
Columns Names in SYSTEM.CATALOG table:-
Query:-
“select
TABLE_NAME,DATA_TABLE_NAME,INDEX_TYPE,INDEX_STATE,INDEX_DISABLE_TIMESTAMP from
system.catalog where INDEX_TYPE is not null;”
TABLE_NAME:-
Name of the index
DATA_TABLE_NAME:-
Name of the parent data table of index
INDEX_TYPE:-
GLOBAL(1)
LOCAL(2)
INDEX_STATE:-
Index States (Abbreviation stored in SYSTEM.CATALOG, states
in bold are available to client too):-
BUILDING("b")
USABLE("e")
UNUSABLE("d")
ACTIVE("a")
INACTIVE("i")
DISABLE("x")
REBUILD("r")
(Below paragraph is from
phoenix site)
DISABLE will cause the no
further index maintenance to be performed on the index and it will no longer be
considered for use in queries.
REBUILD will completely rebuild
the index and upon completion will enable the index to be used in queries
again.
BUILDING will partially rebuild the
index from the last disabled timestamp and upon completion.
INACTIVE/UNUSABLE will
cause the index to no longer be considered for use in queries, however index
maintenance will continue to be performed.
ACTIVE/USABLE will
cause the index to again be considered for use in queries.
Note that a disabled index must be rebuild and
cannot be set as USABLE
INDEX_DISABLE_TIMESTAMP:-
It is the timestamp at which index is disabled. It will be 0
, if the index is active or disabled by client manually and will non-zero when
index is disabled during write failures.
Automatic Rebuild process:-
MetaDataRegionObserver is responsible of running
rebuild thread, so upsert select query to update the disabled index is executed
from the regionserver , which is hosting SYSTEM.CATALOG table.
INACTIVE and DISABLE indexes are chosen for
rebuild( provided all the regions of index table are online)
All Indexes of all tables are built serially.
We
build the index from disabled timestamp - phoenix.index.failure.handling.rebuild.overlap.time(default
5 minutes) to SCN
Upsert
select query used is "UPSERT /*+ NO_INDEX */ INTO index_table_name(indexedCols)
select dataCols from data_table”