Support Questions

Find answers, ask questions, and share your expertise

error creating a partitioned parquet table in impala shell? how to use "partitioned by" exactly?

avatar
New Contributor

Hi,

I'm trying to create a partitioned parquet table based on an existing table. I'll create a partition column "txn_date" based on a timestamp string column in the original table. Below is my code:

create table if not exists new_table_part_by_txndate partitioned by (txn_date string) stored as parquet location '/sandbox/my_dir/new_table_part_by_txndate' as (select *, substr(time_stamp_col, 1,10) as txn_date from original_table);

 

but I get error: "ERROR: AnalysisException: Syntax error in line 1: ......Encountered: AS Expected"

 

I then tried to create the table first without loading data:

create table if not exists new_table_part_by_txndate like original_small_with_txndate partitioned by (txn_date string) sotred as parquet location '/sandbox/my_dir/new_table_part_by_txndate';

 

then I get error: "ERROR: AnalysisException: Syntax error in line 1:.......Encountered: PARTITIONED Expected: COMMENT, LOCATION, STORED"

 

please help

1 REPLY 1

avatar
New Contributor

ok I think I might know the reason. my impala version is 2.12, and according to documentation "Prior to CDH 5.7 / Impala 2.5, you could use a partitioned table as the source and copy data from it, but could not specify any partitioning clauses for the new table.".

 

So it appears that my impala is too old to support creating partitioned table while only support reading an existing partitioned table...