WARNINGS: Error applying Kudu Op.: Incomplete: buffer size limit is too small to fit operation

Question

While creating table , its failing with below error.

Query: CREATE TABLE REPORTKUDU
PRIMARY KEY(Marketing_Cloud_Visitor_ID)
PARTITION BY HASH(Marketing_Cloud_Visitor_ID) PARTITIONS 10
STORED AS KUDU TBLPROPERTIES('kudu.master_addresses'='X.X.X.X','kudu.num_tablet_replicas'='1')
AS SELECT * FROM REPORTN

WARNINGS: Error applying Kudu Op.: Incomplete: buffer size limit is too small to fit operation: required 52236317, size limit 10485760

Key already present in Kudu table 'impala::db.REPORTKUDU'. (1 of 9006 similar)

kunzhao · Accepted Answer

Impala Command Line Argument Advanced Configuration Snippet (Safety Valve)
-kudu_mutation_buffer_size=20971520
-kudu_error_buffer_size=20971520

Tablet Server Advanced Configuration Snippet (Safety Valve) for gflagfile
-max_cell_size_bytes=20971520

=========================

As above setted，It is working fine.

Thanks .

Grant Henke · Answer

I suspect that large number of "Key already present" errors may play a part in the initial buffer size warning.

The `Key already present in Kudu table 'impala::db.REPORTKUDU'. (1 of 9006 similar)` is telling you that

you are trying to insert records which are not unique. The table you are selecting from must have multiple records for each `Marketing_Cloud_Visitor_ID`. I would suggest writing the create table statement to ensure the primary key is unique, or adjusting the select statment to deduplicate the records before inserting into Kudu.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.