Thanks for posting the resolution. I'm curious, though, was this a Kudu table or a table on HDFS? ... View more

Hey @RikG, you are correct.  The way errors are communicated is dependent on the configured FlushMode.  In your example the FlushMode is not set, so it's the default of AUTO_FLUSH_SYNC.  In AUTO_FLUSH_SYNC mode any per-row errors are immediately returned as part of the OperationResponse, since the write happens synchronously.  In AUTO_FLUSH_BACKGROUND mode it's necessary to call getPendingErrors.  The docs on AUTO_FLUSH_BACKGROUND cover this to some extent. ... View more

Hi RikG, what do you mean by passing fewer arguments to the Kudu API?  Note that for row errors you must call KuduSession.getPendingErrors (https://kudu.apache.org/apidocs/org/apache/kudu/client/KuduSession.html#getPendingErrors--). ... View more

The ACL is a whitelist, so you can add whichever users you like.  The default value is '*', which allows any authenticated user to be authorized, but if you want to limit it to joe, bob, and impala you can set it to 'joe,bob,impala'.  In this case joe and bob can take whatever actions they like directly on Kudu (including through Spark) and all other users going through Impala will be subject to Impala's more fine-grained authorizations.   The Kudu CLI is subject to the exact same authorization checks as any other application interacting with Kudu, including direct client access, Spark and Impala.  To Kudu, it's 'just another' client. ... View more

@Brazwrote: After testing, we've found that this is through the Kudu command-line interface where these coarse-grained ACLs get applied. The ACLs are enforced by th server, not the kudu CLI tool itself. @Brazwrote: We observed a non-admin user had access to drop tables in a database that this account had not been granted access to. Right, as discussed in the docs you linked, user principals have 'the ability to create, drop, and alter tables, as well as read, insert, update, and delete data'.  If you would like to restrict a user from being able to drop a table, you will need to remove them from the user ACL whitelist.  Note that this will also disallow them from all other interactions with Kudu data.  A common configuration is to only include the 'impala' user in the user ACL whitelist, in which case only users authenticated through Impala will be able to use Kudu.  Impala will apply its own Sentry-based authorizations which can distinguish privileges at a finer level. ... View more

Hi AKB, there's no table-level compression.  The closest equivalent would be specifying compression for all columns.  It's pretty typical to rely on encodings to reduce size as much as possible, and then additionally apply compression only for particularly big or cold columns. ... View more

Hi AKB,   Nothing is expected to show up in Kudu logs, because the failure is occurring in the Impala query analysis stage.  I haven't been able to reproduce something similar using CDH 5.14, for instance the following queries complete successfully:   CREATE TABLE fact_patientencounter ( pk_PatientEncounterID int, fk_AgencyID int, measure_ChuteTime_TimeInSeconds bigint, measure_Response_TimeInSeconds bigint ) STORED AS PARQUET; CREATE TABLE fact_patientencounter_kudu PRIMARY KEY (pk_PatientEncounterID, fk_AgencyID) PARTITION BY HASH(fk_AgencyID) PARTITIONS 3 STORED AS KUDU AS SELECT pk_PatientEncounterID, fk_AgencyID FROM fact_patientencounter; @tmarshalldo you have any ideas about where that null analysis exception might be coming from? ... View more

Hard to say for certain without the error message, but it's likely that the parser doesn't like the column types being supplied in the PRIMARY KEY clause. ... View more