Reply
Highlighted
New Contributor
Posts: 3
Registered: ‎08-30-2018
Accepted Solution

Kudu 300 columns limitation

Hello,
 
We have notice that Kudu has a limitation of 300 columns per table, but unfortunately we have a table with 396 columns in our system, which violates this. We already tried to increase the column limit to 400 and ingest data into that table and haven't notice any issues and the performance looked good for us.
 
What is the underlying reason for this limit? Could you please share some information about the known implications of exceeding this limit and the circumstances in which they can become more apparent? Do you have an example of test case (be it regarding performance or functional) which fails due to the use of tables with more than 300 columns?
 
Is it in the Kudu roadmap to provide support for higher number of columns in the future? Are the any workarounds to use Kudu with more than 300 columns besides setting a higher maximum value?
 
Thanks,
Paulo.
Cloudera Employee
Posts: 15
Registered: ‎11-12-2013

Re: Kudu 300 columns limitation

Hey Paulo,

 

I saw you posted the same question to the Apache Kudu users mailing list. Here's the response I wrote there:

Check out this older post by Todd Lipcon about the 300 column limit:

http://mail-archives.apache.org/mod_mbox/kudu-user/201706.mbox/%3CCADY20s7iT7%2BrVZNhagnNFUjk7-nNMxJK6%2BnHV%2B2SzpHXKFxvmw%40mail.gmail.com%3E

There are probably other folks who run with over 300 columns in their
schemas, but it's not something Kudu developers are actively testing.
You're certainly welcome to try it, and I'm pleased that your 400
column schema is working well for you, but just be aware that you may
run into unforeseen issues.
New Contributor
Posts: 3
Registered: ‎08-30-2018

Re: Kudu 300 columns limitation

Hi there,

Thanks a lot for the answer.

Announcements