Member since
09-17-2020
8
Posts
0
Kudos Received
0
Solutions
12-07-2020
02:41 AM
Hi, community! I am exploring Kudu - Impala interaction and I can't find a good way to secure kudu table from impala. Let's say, I have Kudu table "test" created from CLI. Then, I login into impala-shell with user "u1", which have an access to database "db1" and create table in this database like this: CREATE EXTERNAL TABLE db1.test_kudu1 STORED AS kudu TBLPROPERTIES ('kudu.table_name' = 'test'); I logout from "u1" user and login as "u2" user which does not have access to database db1, but do have access to database "db2" and this user create table in the "db2" database same way "u1" created table db1.test_kudu1: CREATE EXTERNAL TABLE db2.test_kudu2 STORED AS kudu TBLPROPERTIES ('kudu.table_name' = 'test'); Now, both users have access to kudu table "test", they both can modify this table, delete and insert data. Is there any way to avoid such insecurity with kudu tables? It seems like if we gain access to kudu tables to impala service user, any impala user that have access to creation tables in any database can create table above kudu table and then modify it. Is it really true?
... View more
Labels:
- Labels:
-
Apache Impala
-
Apache Kudu
12-06-2020
11:46 PM
Yeah, that was my bad. However, this didn't help. I want to get warnings that I get from impala-shell when warning like below happens. Or at least something that can indicate these warnings to process it properly. WARNINGS: Error converting column: 0 to DECIMAL(38,15) Error parsing row: file: hdfs://hdpmigr.gaz.ru:8020/db1/t1/t1.csv, before offset: 52 Method getWarnings() doesn't help, probably there is another method to get it
... View more
12-02-2020
03:21 AM
Hello, everyone! I have encountered an issue with obtaining warning from query from my app. Let's say I have a table in impala with DDL like this: CREATE TABLE db1.t1 ( id int, name string ) ROW FORMAT DELIMITED FIELDS TERMINATED BY "," STORED AS TEXTFILE LOCATION '/db1/t1'; Than I put CSV file below in table directory: 1,sdf dasd,2 After I do "INVALIDATE METADATA" and "SELECT * FROM db1.t1", I got several warnings: WARNINGS: Error converting column: 0 to INT Error parsing row: file: hdfs://hdpmigr.gaz.ru:8020/db1/t1/t1.csv, before offset: 21 I want to get these warnings from my app using JDBC driver. Here is a simple class, where I want to print warnings above: import java.sql.*; public class Tst { public static void main(String[] args) { try(Connection conn = DriverManager.getConnection("jdbc:impala://127.0.0.1:21050"); Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("select * from db1.t1")) { System.out.println(conn.getWarnings()); System.out.println(stmt.getWarnings()); System.out.println(rs.getWarnings()); while(rs.next()) System.out.println(rs.getInt("id")); } catch (SQLException throwables) { throwables.printStackTrace(); } } } All three getWarnings print "null". The question is can I somehow get warnings which I receive from impala-shell?
... View more
Labels:
- Labels:
-
Apache Impala
10-15-2020
03:24 AM
After careful exploration of logs I have found some strange behavior and cant explain it. After restart there are maximum 2 vcores, that can be allocated for this queue. However, when I launch jobs, I see more vcores allocated than max for this queue. Screenshots below are made one after another. Why does it happen?
... View more
10-15-2020
12:11 AM
I am running a cluster with Capacity Scheduler. Can I limit vcores for a queue with concrete number of vcores, not percentages? I found option yarn.scheduler.capacity.<queue_path>.maximum-allocation-vcores. But it seems like it doesn't work. I added this option into "Capacity Scheduler Configuration Advanced Configuration Snippet (Safety Valve)" in Cloudera Manager, but applications still use more than this restriction. Can someone explain why?
... View more
Labels:
- Labels:
-
Apache YARN
09-29-2020
12:34 AM
How should I install impala-udf-devel package? Is it installed along with another services?
... View more
Labels:
- Labels:
-
Apache Impala