Member since
12-10-2018
23
Posts
0
Kudos Received
0
Solutions
02-22-2019
12:58 AM
Hi. I user Claudera Manager to install Sentry, but can't. Can't open /var/run/cloudera-scm-agent/process/1747-sentry-SENTRY_SERVER/supervisor_status: Permission denied. Looks, i have no premissions. sudo ls -lah /var/run/cloudera-scm-agent/process/1747-sentry-SENTRY_SERVER
total 64K
drwxr-x--x 3 sentry sentry 360 Feb 22 11:26 .
drwxr-x--x 161 sentry root 3.2K Feb 22 11:26 ..
-rw-r----- 1 sentry sentry 250 Feb 22 11:26 cloudera-monitor.properties
-rw-r----- 1 sentry sentry 342 Feb 22 11:26 cloudera-stack-monitor.properties
-rw------- 1 root root 5.1K Feb 22 11:26 config.zip
-rw-r----- 1 sentry sentry 3.3K Feb 22 11:26 core-site.xml
-rw-r----- 1 sentry sentry 511 Feb 22 11:26 creds.localjceks
-rw------- 1 sentry sentry 1 Feb 22 11:26 exit_code
drwxr-x--x 2 sentry sentry 80 Feb 22 11:26 logs
-rw-r----- 1 sentry sentry 579 Feb 22 11:26 navigator.client.properties
-rw------- 1 root root 3.1K Feb 22 11:26 proc.json
-rw-r----- 1 sentry sentry 1.7K Feb 22 11:26 redaction-rules.json
-rw------- 1 sentry sentry 762 Feb 22 11:26 sentry.keytab
-rw-r----- 1 sentry sentry 2.4K Feb 22 11:26 sentry-log4j.properties
-rw-r----- 1 sentry sentry 2.5K Feb 22 11:26 sentry-site.xml
-rw-r----- 1 sentry sentry 315 Feb 22 11:26 ssl-client.xml
-rw------- 1 root root 3.5K Feb 22 11:26 supervisor.conf
-rw------- 1 root root 636 Feb 22 11:26 supervisor_status So, i don't know how to solve this issue. But i see two ways: 1. Runs C-Manager as sudo 2. something with paths - any way to create its as sentry-user. PS: or, i confused in users/roles. I don't know what linux-user starts C-Manager, what linux-user runs sentry installation. And how about roles (in Manager)? How it correlated with linux users?
... View more
Labels:
- Labels:
-
Apache Sentry
-
Cloudera Manager
02-19-2019
01:01 AM
Hi. I need to import data from non-trivial dbms: SAS (it isn't db at all)... So, i have jdbc-driver (class "com.sas.rio.MVADriver"), and i have socket (host and port). And i have no connection-manager at all. What should i do? How to write working import-string? 1. What about connection-manager 2. "driver" looks like jdbc:mysql://<ip>:<port>/ So "mysql" is strange part of it. How to replace it? 3. Where must i put drives (jar-files) to use it from sqoop?
... View more
Labels:
- Labels:
-
Apache Impala
-
Apache Oozie
-
Apache Sqoop
02-19-2019
12:45 AM
@bgrabows wrote: Did you try: refresh <db_name>.<table_name> ? Regards, Bart Of course, i did. And... nothing happens. I think issue more basicly then searching problems. I can't set new user any access for select or insert any tables. It looks like my Sentry brocken, and any new accessable action can't run at all.
... View more
02-13-2019
09:45 PM
It's simple... There're no "hdfs group" - u mast assign user and group in host-os. And, if u have a cluster, u mast do it at "main node" (or at every node if "main" migrate to another server). But it doen't solve my core problem: "need to create new user and grant to it few rules (on select of couple schemas)." So, user in group, group have premissions, but there are no effect of it. What must i do in this case?
... View more
02-13-2019
06:57 AM
Hi, i need to create new user and grant to it few rules (on select of couple schemas). i create group, create user, assign user to group, and see it in hdfs - and there're no groups. 1. sudo adduser tezd_user 2. sudo groupadd tezd_group 3. sudo usermod -a -G tezd_group tezd_user 4. groups tezd_user >> tezd_user : tezd_user tezd_group 5. hdfs groups tezd_user >> tezd_user : So, user have no group - have to rules - have no access to any action in db. How to set group for user, and how to get rules to user (i think, it makes by addition group to user)?
... View more
Labels:
- Labels:
-
Apache Impala
-
Cloudera Hue
-
HDFS
-
Kerberos
02-12-2019
05:37 AM
"Do both of these assert the right values you've set? SHOW CURRENT ROLES; SHOW GRANT ROLE developer;" And if i set role to group tezd_group. Create new user tezd_user, assign tezd_user to tezd_group. And see "SHOW CURRENT ROLES;" -> 0 rows? So, it looks like user have no groups, or something like this... Where's the problem?
... View more
01-31-2019
06:53 AM
I used all of this three buttons, but there's no effect on searching tables.
... View more
01-31-2019
05:06 AM
Hi. Have an issue: new tables can't find by search. What's wrong? Any metadata update, refresh schema function, set table property, anything? Please, "let's Dasha find impala tables".
... View more
Labels:
- Labels:
-
Apache Impala
-
Cloudera Hue
01-16-2019
11:16 PM
Metainfo "partitions" mast have information about rules for partition (what column, size of part), not "partition instances". The same about index - need info about tuple of columns included into index.
... View more
01-16-2019
08:48 AM
Ok, thanks. But we just kill our kudu-server. So we can use kudu only from Impala.
... View more
01-11-2019
04:45 AM
Hi, i need to collect information like "create table" statment: table and schema names, columns and its types, comments and so on. So, i'm able to collect table schema and table names and its location ("TBLS"), columns ("COLUMNS_V2") and table prooperties ("TABLE_PARAMS"). But i can't get any info about PK and partitions. Tables "PARTITIONS", "PARTITION_KEYS", "IDXS", "INDEX_PARAMS" are empty. So, i don't know where can i find information about. Where's it? Of course, i create table with partitions and primary keys. Oh, and i should be get type of table too: parquete, textfile or kudu-table. Havn't this info too.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
Apache Kudu
01-11-2019
03:59 AM
Hi. Where's KUDU stores own tables? I create testing kudu-table CREATE TABLE impala_kudu.krg_test_table (
id int not nULL,
col1 int NULL,
col2 STRING NULL,
col3 float COMMENT 'only for test',
PRIMARY KEY (id)
)
PARTITION BY HASH (id) PARTITIONS 10
COMMENT 'new testing kudu table'
STORED AS KUDU
TBLPROPERTIES ('test_arg'='test_value1','test_arg2'='test_value','comment'='arg comment','comment1'='arg comment','kudu.master_addresses'='hozd8:7051', 'kudu.num_tablet_replicas' = '1'); Then insert into it some data: insert into impala_kudu.krg_test_table
values(1,14,"qql",3.7);
insert into impala_kudu.krg_test_table
values(2,59,"adasdasd",6); Now i see in "hive metastore" table location: it shows me next hdfs://hozd8:8020/user/hive/warehouse/impala_kudu.db/krg_test_table But such file-path is empty! Where's data? Where kudu stores it? PS: i need to admin partitions - add it, drop it and so on... But i don't know where it stores, so i can't do anything.
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
Apache Kudu
12-18-2018
08:26 AM
Hi. Physicly impala table stores as planty of files. Does any methods to manage them? To set file-size? Any attribute?
... View more
Labels:
- Labels:
-
Apache Hive
-
Apache Impala
-
HDFS
12-18-2018
08:23 AM
Look, at first input-box i could set whole path to script - and that's all. Why we need file-property, and exec-property?
... View more
12-17-2018
12:46 AM
Very thanks. It works. But it looks strange. Why i need to set two path: real path, and second one (logic path?), what purpose does it have?
... View more
12-11-2018
01:11 AM
yes, it is. Not me, sysadmin. If i configed impala - i'd know correct port.
... View more
12-10-2018
10:18 PM
Hi, thnx. It was port-error. But it's not 21000, it works on 23005 port. I don't know how and why - but it is.
... View more
12-10-2018
05:59 AM
Hi.
I run Impala-shell command
impala-shell -i hdp01.***.***.ru:21051 -k -f /tmp/qql.sql
and catch such error:
Starting Impala Shell using Kerberos authentication
Using service name 'impala'
Error connecting: TTransportException, TSocket read 0 bytes
Not connected to Impala, could not execute queries.
That's all.
No logs. No special error description. Noghing. Just "not connected to Impala".
However, Impala works fine. In Hue i can exec queries.
And "host:port" is fine. If i set other combination of it, i get:
Starting Impala Shell using Kerberos authentication Using service name 'impala' Error connecting: TTransportException, Could not connect to any.other.port.ru:21050
So, we can see, that server available. But Impala is not.
Why? And how to fix it?
I even can't google any solution by my error: "Error connecting: TTransportException, TSocket read 0 bytes" or "Not connected to Impala, could not execute queries."
Please, help me.
... View more
Labels:
- Labels:
-
Apache Impala
-
Cloudera Hue
-
Kerberos
12-10-2018
03:49 AM
Hi. I'd have a problem: need to run bash-script from oozie, but i can't. Ther're an error: Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], main() threw exception, Cannot run program "ref_tbl.sh" (in directory "/data6/yarn/nm/usercache/hdfs-etl/appcache/application_1544099183263_0655/container_e295_1544099183263_0655_01_000002"): error=2, No such file or directory
java.io.IOException: Cannot run program "ref_tbl.sh" (in directory "/data6/yarn/nm/usercache/hdfs-etl/appcache/application_1544099183263_0655/container_e295_1544099183263_0655_01_000002"): error=2, No such file or directory So: 1. What directory should i put my bash-script? Relative and absolute path, please. 2. How can i find my workflow? 3. I used "oozie editor", not edit xml-file by hands. So i can't use solution like "http://crazyadmins.com/tag/how-to-execute-shell-script-in-oozie/ Step 3: Create workflow.xml file for your shell action. ") Please, help me.
... View more
Labels:
- Labels:
-
Apache Oozie
12-10-2018
02:07 AM
Hi. How can you find your workflow from list of all workflows? They are nameless...
... View more
12-10-2018
02:03 AM
Hi. Please, tell me how to find any WF from all wf-list in Hue's file browser? I have name and id of my workflow. I go to "/user/hue/oozie/workspaces" and try to find it, but... There are only strange names like "hue-oozie-1503584643.04": it's not name or id. So i can't find my wf in this nameless heep of WFs. How can i find it?
... View more
Labels:
- Labels:
-
Apache Oozie
-
Cloudera Hue