Skip to main content
ipson
ipson
Gold ⭐️
February 13, 2024
Solved

Adding new column to iceberg table silently failing

I have an existing iceberg table and I'm trying to evolve schema by adding a new column:

`alter table my_db.my_table add column insert_time timestamp`

When I execute the above statement, I get a 'New column(s) have been added to the table.' but the column doesn't appear. I tried running a REFRESH, an INVALIDATE METADATA, and I tried restarting Impala but it had no effect. I tried the same operation on a test table and it worked. I'm using Impala as a query engine and Azure for storage. Impala is running in a Data Hub Cluster.

    jAnshula
    jAnshulaAnswer
    Gold ⭐️
    February 13, 2024

    Hi @ipson 

    Please check table schema and verify if ('external.table.purge'='true') property is set using S

    This property controls how DROP TABLE and ALTER TABLE works.

    In case it is not present, then add this property for the required table( issue the ALTER TABLE stmt twice)

    ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');
    ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');

    The first invocation sets the property in HMS. The second one persists it to Iceberg metadata.

    Let me know if this helps.

    View in thread

    1 reply

    jAnshula
    Gold ⭐️
    jAnshulaAnswer
    Gold ⭐️
    February 13, 2024

    Hi @ipson 

    Please check table schema and verify if ('external.table.purge'='true') property is set using S

    This property controls how DROP TABLE and ALTER TABLE works.

    In case it is not present, then add this property for the required table( issue the ALTER TABLE stmt twice)

    ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');
    ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');

    The first invocation sets the property in HMS. The second one persists it to Iceberg metadata.

    Let me know if this helps.

    Translate
      Terms & ConditionsAccessibility statement