WARNINGS: Error applying Kudu Op.: Incomplete: buffer size limit is too small to fit operation

[#1]

While creating table , its failing with below error.

Query: CREATE TABLE REPORTKUDU
PRIMARY KEY(Marketing_Cloud_Visitor_ID)
PARTITION BY HASH(Marketing_Cloud_Visitor_ID) PARTITIONS 10
STORED AS KUDU TBLPROPERTIES('kudu.master_addresses'='X.X.X.X','kudu.num_tablet_replicas'='1')
AS SELECT * FROM REPORTN

WARNINGS: Error applying Kudu Op.: Incomplete: buffer size limit is too small to fit operation: required 52236317, size limit 10485760

Key already present in Kudu table 'impala::db.REPORTKUDU'. (1 of 9006 similar)

[#2]

I suspect that large number of "Key already present" errors may play a part in the initial buffer size warning.

The `Key already present in Kudu table 'impala::db.REPORTKUDU'. (1 of 9006 similar)` is telling you that

you are trying to insert records which are not unique. The table you are selecting from must have multiple records for each `Marketing_Cloud_Visitor_ID`. I would suggest writing the create table statement to ensure the primary key is unique, or adjusting the select statment to deduplicate the records before inserting into Kudu.

[#3]

I encountered the same problem when performing insert operation. Have you solved this problem? thank you

[#4]

Hi @rahul_gaikwad,

The issue occurs due to a known limitation. As the code points out, it indicates that the single write operation cannot fit into the configured maximum buffer size.Please refer to this KB:

https://my.cloudera.com/knowledge/quot-ERROR-Error-applying-Kudu-Op-Incomplete-buffer-size?id=302775

Regards,

Will

[#5 · Accepted solution]

Impala Command Line Argument Advanced Configuration Snippet (Safety Valve)
-kudu_mutation_buffer_size=20971520
-kudu_error_buffer_size=20971520

Tablet Server Advanced Configuration Snippet (Safety Valve) for gflagfile
-max_cell_size_bytes=20971520

=========================

As above setted，It is working fine.

Thanks .