question Re: Add new Columns in Kudu Tables from Impala in Support Questions

Add new Columns in Kudu Tables from Impala

Guarupe — Thu, 01 Jul 2021 17:46:53 GMT

Hi Folks,
After adding new columns in Kudu tables from impala, I had some issues computing stats for this table. Everything works fine until the Compute Stats command;

The alter table command

ALTER TABLE myschema.mytable ADD COLUMNS (mycolumn TIMESTAMP DEFAULT now());

The issue when Compute Stats command.

ImpalaRuntimeException: Error making 'updateTableColumnStatistics' RPC to Hive Metastore: CAUSED BY: MetaException: Column mycolumn doesn't exist in table mytable in database myschema

Someone can help me, please?

Re: Add new Columns in Kudu Tables from Impala

Shelton — Sun, 04 Jul 2021 11:07:51 GMT

@Guarupe
I responded to a similar question Warm up Impala
You will need to run the INVALIDATE METADATA [[db_name.]table_name]
The error is precise Impala uses the Hive Metastore [HMS] to build efficient queries

CAUSED BY: MetaException: Column mycolumn doesn't exist in table mytable in database myschema

In your case INVALIDATE METADATA [[myschema.]mytable]
The INVALIDATE METADATA is an asynchronous operation that simply discards the loaded metadata from the catalog and coordinator caches. After that operation, the catalog and all the Impala coordinators only know about the existence of databases and tables and nothing more. Metadata loading for tables is triggered by any subsequent queries.

After running this in the impala-shell you should compute statistics successfully

Happy hadooping

Re: Add new Columns in Kudu Tables from Impala

Guarupe — Mon, 05 Jul 2021 12:08:34 GMT

Hi Shelton,
Thanks for your reply, but INVALIDATE METADATA didn't work for me. I still have the same error. My option was to recreate the table with a CREATE TABLE as SELECT from the previous table.
Thanks again

Re: Add new Columns in Kudu Tables from Impala

Shelton — Mon, 05 Jul 2021 13:08:18 GMT

@Guarupe
Can you share your steps, please? Are you suing HUE to run your cmds? Did you use the Impala editor?

Re: Add new Columns in Kudu Tables from Impala

Guarupe — Mon, 05 Jul 2021 14:13:25 GMT

Hi Shelton,

The answer to all questions is yes. I'm using the Hue Impala Editor and all these commands were issued by HUE. As I've already solved the problem with a CREATE AS SELECT, I'm going to ask for a little time to simulate this problem again.

Thanks again,

Re: Add new Columns in Kudu Tables from Impala

Guarupe — Tue, 29 Mar 2022 19:17:00 GMT

Hi @Shelton, sorry for the delay in responding.
I finally found the solution to my problem. I need to specify the column properties and then Compute Stats will work perfectly.
```sql
ALTER TABLE myschema.mytable
ADD COLUMNS (mycolumn1 TIMESTAMP NULL COMPRESSION DEFAULT_COMPRESSION,
mycolumn2 TIMESTAMP NULL COMPRESSION DEFAULT_COMPRESSION);
```