Adding new column to iceberg table silently failing

[#1]

I have an existing iceberg table and I'm trying to evolve schema by adding a new column:

`alter table my_db.my_table add column insert_time timestamp`

When I execute the above statement, I get a 'New column(s) have been added to the table.' but the column doesn't appear. I tried running a REFRESH, an INVALIDATE METADATA, and I tried restarting Impala but it had no effect. I tried the same operation on a test table and it worked. I'm using Impala as a query engine and Azure for storage. Impala is running in a Data Hub Cluster.

[#2 · Accepted solution]

Hi @ipson 

Please check table schema and verify if ('external.table.purge'='true') property is set using S

This property controls how DROP TABLE and ALTER TABLE works.

In case it is not present, then add this property for the required table( issue the ALTER TABLE stmt twice)

ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');
ALTER TABLE t SET TBLPROPERTIES ('external.table.purge'='true');

The first invocation sets the property in HMS. The second one persists it to Iceberg metadata.

Let me know if this helps.