Hi, community! I am exploring Kudu - Impala interaction and I can't find a good way to secure kudu table from impala.   Let's say, I have Kudu table "test" created from CLI. Then, I login into impala-shell with user "u1", which have an access to database "db1" and create table in this database like this: CREATE EXTERNAL TABLE db1.test_kudu1 STORED AS kudu TBLPROPERTIES ('kudu.table_name' = 'test'); I logout from "u1" user and login as "u2" user which  does not have access to database db1, but do have access to database "db2" and this user create table in the "db2" database same way "u1" created table db1.test_kudu1:   CREATE EXTERNAL TABLE db2.test_kudu2 STORED AS kudu TBLPROPERTIES ('kudu.table_name' = 'test');   Now, both users have access to kudu table "test", they both can modify this table, delete and insert data. Is there any way to avoid such  insecurity with kudu tables?   It seems like if we gain access to kudu tables to impala service user, any impala user that have access to creation tables in any database can create table above kudu table and then modify it. Is it really true? ... View more

Hello, everyone! I have encountered an issue with obtaining warning from query from my app. Let's say I have a table in impala with DDL like this: CREATE TABLE db1.t1 ( id int , name string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY "," STORED AS TEXTFILE LOCATION '/db1/t1' ; Than I put CSV file below in table directory: 1 ,sdf dasd, 2 After  I  do "INVALIDATE METADATA" and "SELECT * FROM db1.t1", I got several warnings: WARNINGS : Error converting column : 0 to INT Error parsing row : file : hdfs : //hdpmigr.gaz.ru:8020/db1/t1/t1.csv, before offset: 21 I want to get these warnings from my app using JDBC driver.  Here is a simple class, where I want to print warnings above: import java.sql. * ; public class Tst { public static void main ( String [] args ) { try ( Connection conn = DriverManager . getConnection ( "jdbc:impala://127.0.0.1:21050" ); Statement stmt = conn. createStatement (); ResultSet rs = stmt. executeQuery ( "select * from db1.t1" )) { System . out . println (conn. getWarnings ()); System . out . println (stmt. getWarnings ()); System . out . println (rs. getWarnings ()); while (rs. next ()) System . out . println (rs. getInt ( "id" )); } catch ( SQLException throwables ) { throwables . printStackTrace (); } } } All three getWarnings print "null". The question is can I somehow get warnings which I  receive from impala-shell?  ... View more

Hello to everyone.   After upgrading to JDBC42-2.6.17 impala driver I start getting some error messages from log4j while connecting to Impala. Here simple app to show the problem: import org.apache.log4j.LogManager ; import org.apache.log4j.Logger ; import java.sql. Connection ; import java.sql.DriverManager ; import java.sql.SQLException ; import java.sql. Statement ; public class Tst { public static void main ( String [] args ) { Logger log = LogManager . getRootLogger (); log. error ( "Tst" ); try ( Connection conn = DriverManager . getConnection ( "jdbc:impala://127.0.0.1:21050" ); Statement stmt = conn. createStatement ()) { stmt. execute ( "select 1" ); } catch ( SQLException throwables ) { throwables . printStackTrace (); } } } My log4j.propeties file: log4j.rootLogger = DEBUG, console1 log4j.appender.console1 = org.apache.log4j.ConsoleAppender log4j.appender.console1.layout = org.apache.log4j.PatternLayout  Command to launch: /usr/java/ jdk1 .8.0_232 - cloudera / bin / java - cp tst_log4j - 1.0 - SNAPSHOT.jar : log4j - 1.2.17 .jar : ImpalaJDBC42.jar - Dlog4j.configuration = file :/ home / dl_efw / tst / log4j.properties Tst Output: Tst log4j : ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "com.cloudera.impala.jdbc42.internal.apache.log4j.Appender" variable. log4j : ERROR The class "com.cloudera.impala.jdbc42.internal.apache.log4j.Appender" was loaded by log4j : ERROR [sun.misc.Launcher$AppClassLoader@ 15d b9742] whereas object of type log4j : ERROR "org.apache.log4j.ConsoleAppender" was loaded by [sun.misc.Launcher$AppClassLoader@ 15d b9742]. log4j : ERROR Could not instantiate appender named "console1" . log4j : WARN No appenders could be found for logger (com.cloudera.impala.jdbc42.internal.apache.thrift.transport.TSaslTransport). log4j : WARN Please initialize the log4j system properly. log4j : WARN See http : //logging.apache.org/log4j/1.2/faq.html#noconfig for more info. How can I remove this messages? Did someone encounter such a problem? Impala has default log4j settings. With previous JDBC driver there was no problem. Any help appreciated! ... View more

I am running a cluster with Capacity Scheduler. Can I limit vcores for a queue with concrete number of vcores, not percentages? I found option yarn.scheduler.capacity.<queue_path>.maximum-allocation-vcores. But it seems like it doesn't work. I added this option into " Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)" in Cloudera Manager, but applications still use more than this restriction. Can someone explain why? ... View more

When launching query in Hive, which use MapReduce to execute query, for example: ``` SELECT COUNT(*) FROM <TABLE_NM>; ``` I got next messages and then query hang. In yarn application list there is no jobs at all after that. What can cause such a problem? Which properties should I look at? I am newbie here. ``` INFO : Compiling command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f): select count(*) from efw_wrk.tst_11 INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f); Time taken: 0.073 seconds INFO : Executing command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f): select count(*) from efw_wrk.tst_11 INFO : Query ID = hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode ``` ... View more