Kudu 300 columns limitation

[#1]

Hello,

We have notice that Kudu has a limitation of 300 columns per table, but unfortunately we have a table with 396 columns in our system, which violates this. We already tried to increase the column limit to 400 and ingest data into that table and haven't notice any issues and the performance looked good for us.

What is the underlying reason for this limit? Could you please share some information about the known implications of exceeding this limit and the circumstances in which they can become more apparent? Do you have an example of test case (be it regarding performance or functional) which fails due to the use of tables with more than 300 columns?

Is it in the Kudu roadmap to provide support for higher number of columns in the future? Are the any workarounds to use Kudu with more than 300 columns besides setting a higher maximum value?

Thanks,

Paulo.

[#2 · Accepted solution]

Hey Paulo,

I saw you posted the same question to the Apache Kudu users mailing list. Here's the response I wrote there:

Check out this older post by Todd Lipcon about the 300 column limit:

http://mail-archives.apache.org/mod_mbox/kudu-user/201706.mbox/%3CCADY20s7iT7%2BrVZNhagnNFUjk7-nNMxJK6%2BnHV%2B2SzpHXKFxvmw%40mail.gmail.com%3E

There are probably other folks who run with over 300 columns in their
schemas, but it's not something Kudu developers are actively testing.
You're certainly welcome to try it, and I'm pleased that your 400
column schema is working well for you, but just be aware that you may
run into unforeseen issues.

[#3]

Hi there,

Thanks a lot for the answer.

[#4]

Where can I change this 300 limit