Member since
09-17-2020
8
Posts
0
Kudos Received
0
Solutions
12-07-2020
09:17 AM
These are good questions that come up frequently. https://docs.cloudera.com/runtime/7.2.2/administering-kudu/topics/kudu-security-trusted-users.html discusses the issue. In summary, Hive/Impala tables (i.e. those with entries in the Hive Metastore) are authorized in the same way, regardless of whether backing storage is HDFS, S3, Kudu, HBase, etc - the SQL interface does the authorization to confirm that the end user has access to the table, columns, etc, then the service accesses the storage as the privileged user (Impala in this case). In this model, if you create an external Kudu table in Impala and give permissions to a user to access the table via Impala, then they will have permissions to access the data in the underlying Kudu table. The thing that closes the loophole here is that creating the external Kudu table requires very high privileges - ALL permission on SERVER - a regular user can't create an external Kudu table pointed at an arbitrary Kudu cluster or table.
... View more
12-06-2020
11:46 PM
Yeah, that was my bad. However, this didn't help. I want to get warnings that I get from impala-shell when warning like below happens. Or at least something that can indicate these warnings to process it properly. WARNINGS: Error converting column: 0 to DECIMAL(38,15) Error parsing row: file: hdfs://hdpmigr.gaz.ru:8020/db1/t1/t1.csv, before offset: 52 Method getWarnings() doesn't help, probably there is another method to get it
... View more
12-01-2020
09:45 PM
Hello to everyone. After upgrading to JDBC42-2.6.17 impala driver I start getting some error messages from log4j while connecting to Impala. Here simple app to show the problem: import org.apache.log4j.LogManager ; import org.apache.log4j.Logger ; import java.sql. Connection ; import java.sql.DriverManager ; import java.sql.SQLException ; import java.sql. Statement ; public class Tst { public static void main ( String [] args ) { Logger log = LogManager . getRootLogger (); log. error ( "Tst" ); try ( Connection conn = DriverManager . getConnection ( "jdbc:impala://127.0.0.1:21050" ); Statement stmt = conn. createStatement ()) { stmt. execute ( "select 1" ); } catch ( SQLException throwables ) { throwables . printStackTrace (); } } } My log4j.propeties file: log4j.rootLogger = DEBUG, console1 log4j.appender.console1 = org.apache.log4j.ConsoleAppender log4j.appender.console1.layout = org.apache.log4j.PatternLayout Command to launch: /usr/java/ jdk1 .8.0_232 - cloudera / bin / java - cp tst_log4j - 1.0 - SNAPSHOT.jar : log4j - 1.2.17 .jar : ImpalaJDBC42.jar - Dlog4j.configuration = file :/ home / dl_efw / tst / log4j.properties Tst Output: Tst log4j : ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "com.cloudera.impala.jdbc42.internal.apache.log4j.Appender" variable. log4j : ERROR The class "com.cloudera.impala.jdbc42.internal.apache.log4j.Appender" was loaded by log4j : ERROR [sun.misc.Launcher$AppClassLoader@ 15d b9742] whereas object of type log4j : ERROR "org.apache.log4j.ConsoleAppender" was loaded by [sun.misc.Launcher$AppClassLoader@ 15d b9742]. log4j : ERROR Could not instantiate appender named "console1" . log4j : WARN No appenders could be found for logger (com.cloudera.impala.jdbc42.internal.apache.thrift.transport.TSaslTransport). log4j : WARN Please initialize the log4j system properly. log4j : WARN See http : //logging.apache.org/log4j/1.2/faq.html#noconfig for more info. How can I remove this messages? Did someone encounter such a problem? Impala has default log4j settings. With previous JDBC driver there was no problem. Any help appreciated!
... View more
Labels:
10-15-2020
03:24 AM
After careful exploration of logs I have found some strange behavior and cant explain it. After restart there are maximum 2 vcores, that can be allocated for this queue. However, when I launch jobs, I see more vcores allocated than max for this queue. Screenshots below are made one after another. Why does it happen?
... View more
09-29-2020
03:21 AM
2 Kudos
@Pavel_kostyukov You have to install on the need basis, if you need them then you can install by following below document. https://docs.cloudera.com/documentation/enterprise/latest/topics/impala_udf.html#udf_demo_env
... View more
09-17-2020
02:29 AM
When launching query in Hive, which use MapReduce to execute query, for example: ``` SELECT COUNT(*) FROM <TABLE_NM>; ``` I got next messages and then query hang. In yarn application list there is no jobs at all after that. What can cause such a problem? Which properties should I look at? I am newbie here. ``` INFO : Compiling command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f): select count(*) from efw_wrk.tst_11 INFO : Semantic Analysis Completed (retrial = false) INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null) INFO : Completed compiling command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f); Time taken: 0.073 seconds INFO : Executing command(queryId=hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f): select count(*) from efw_wrk.tst_11 INFO : Query ID = hive_20200917081021_1955a9a6-9fe1-4840-88b4-a9675b87bb2f INFO : Total jobs = 1 INFO : Launching Job 1 out of 1 INFO : Starting task [Stage-1:MAPRED] in serial mode ```
... View more