I would try to eliminate arrays from the design. Maybe remove the array column from the original table and instead have another table with two columns: one references the row of the old table, one contains the array members (one member per row.) This way you can leverage atomicity guarantees which you miss when trying to manipulate arrays. ... View more

This may happen if you use bulkLoad tool on the TEST_TABLE after index creation. ... View more

Have you tried passing a real Object[] array to the update method instead of an ArrayList? Or even easier, use varargs if your example does not need the param ArrayList. jdbctemplatejdbcTemplate.update(sql, param.toArray(newInteger[param.size()])); jdbctemplatejdbcTemplate.update(sql, value1, value2, value3); ... View more

I'm not sure what you wanted to express by listing two storage handlers; but for a Phoenix storage handler example, look at this: https://phoenix.apache.org/hive_storage_handler.html Note that it needs different serdeproperties than HBase does. ... View more

1. The role of stored procedures in big data usually goes to application level. The logic has to be redesigned as a Java program for instance. See also coprocessors in HBase. In particular, Phoenix is designed to execute quick and easy SQL queries, so even if you had no stored procedure but a complex SELECT statement for something in the RDBMS, Phoenix might not support some constructs in it. Consider using Hive with or instead of Phoenix for complex queries. 2. About archiving, first question would be, why would you need it? No disk space? Speed up table scans? There should be a big data way to solve the problem. ... View more

I don't get the philosophical part. Secondary index is orthogonal to what? The secondary index is an INTERSECTION of the primary key and what? ... View more

As for >2 GB blobs, Hive STRING or even BINARY won't handle AFAIK. But that is just googled, Hive experts please add your thoughts. Please note that the "InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit." part in your stack trace tells you that you hit the limits of ProtocolBuffers, not Hive field type limitations. That could explain the 500 MB limit that you got in your investigations. In Hive code, orc input stream implementation I could see that there is 1 GB protobuf limit set but that is for the whole message and the blob is only a part of it. ... View more