Member since
09-04-2018
9
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 6293 | 03-11-2019 08:10 PM |
03-11-2019
08:10 PM
1 Kudo
Sorry, I resolved by myself. By performing backup with the --clean option, restoration was successful. backup [root@localhost ~]$ pg_dump -w -h localhost -p 7432 -U scm --clean > /tmp/scm_server_db_backup.$(date +%Y%m%d) restore [root@localhost ~]$ psql -w -h localhost -p 7432 -U scm -f /tmp/scm_server_db_backup.$(date +%Y%m%d) The above restore was successful.
... View more
03-11-2019
02:08 AM
Hi. I am using Embedded PostgreSQL for CM. Backed up as described in the document below. https://www.cloudera.com/documentation/enterprise/5-16-x/topics/cm_ag_backup_dbs.html#cmig_topic_5_6_3 pg_dump -h hostname -p 7432 -U scm > /tmp/scm_server_db_backup.$(date +%Y%m%d) But trying to restore fails. [root@localhost ~]$ psql -w -h localhost -p 7432 -U scm -f /tmp/scm_server_db_backup.$(date +%Y%m%d) ↓ psql:scm_server_db_backup:54: ERROR: relation "audits" already exists
ALTER TABLE
psql:scm_server_db_backup:77: ERROR: relation "client_configs" already exists
ALTER TABLE
psql:scm_server_db_backup:89: ERROR: relation "client_configs_to_hosts" already exists
ALTER TABLE
.
.
.
psql:scm_server_db_backup:1036: ERROR: duplicate key value violates unique constraint "audits_pkey"
DETAIL: Key (audit_id)=(1) already exists.
CONTEXT: COPY audits, line 1
psql:scm_server_db_backup:1056: ERROR: duplicate key value violates unique constraint "client_configs_pkey"
DETAIL: Key (client_config_id)=(1) already exists.
CONTEXT: COPY client_configs, line 1
psql:scm_server_db_backup:1093: ERROR: duplicate key value violates unique constraint "cluster_activated_releases_pkey"
DETAIL: Key (cluster_id, release_id)=(1, 1) already exists.
.
.
.
psql:scm_server_db_backup:30181: ERROR: multiple primary keys for table "audits" are not allowed
psql:scm_server_db_backup:30189: ERROR: multiple primary keys for table "client_configs" are not allowed
psql:scm_server_db_backup:30197: ERROR: multiple primary keys for table "cluster_activated_releases_aud" are not allowed
.
.
. For the time being, I attempted to execute restoration after dropping all DB. drop_all.sql drop database "postgres";
drop database "scm";
drop database "amon";
drop database "rman";
drop database "nav";
drop database "navms"; But it gets an error. [root@localhost ~]$ psql -U cloudera-scm -p 7432 -h localhost -d postgres -f drop_all.sql
Password for user cloudera-scm: ↓ psql:drop_all.sql:1: ERROR: cannot drop the currently open database psql:drop_all.sql:2: ERROR: database "scm" is being accessed by other users DETAIL: There are 11 other sessions using the database. I tried to drop the process killing of scm database, but the process will be restored while killing. How can I restore Embedded PostgreSQL?
... View more
Labels:
- Labels:
-
Cloudera Manager
03-05-2019
06:24 PM
Thank you for answering. That means that it "cursor.fetchall()" contains hdfs scan time. On the other hand, the bottleneck is not on "hdfs scan" but on the client or network. I checked below, but I interpreted this problem as occurring in the case of specifying a size smaller than the default batch size. https://issues.apache.org/jira/browse/IMPALA-1618 It is questionable whether there is a possibility of occurrence even when using "cursor.fetchall()". I have found an issue that shows the same thing. https://github.com/cloudera/impyla/issues/239 Wes McKinney says it is a problem of hs2client. Somehow I understood that there was no solution.... Thanks!
... View more
03-05-2019
01:12 AM
Hi, There is a program that uses Impyla to retrieve data from the local Impala daemon. cursor.execute("select * from table;")
rows = cursor.fetchall() The table has 5 million rows, the number of columns is 9, the file size at the time of CSV conversion is about 200 MB. There are four data nodes.Memory is 32 GB. Despite just that much data, fetchall () takes over 200 seconds. Query execution ends in 0.2 seconds Why is it so slow? Do you have any ideas to speed up something? Thanks!
... View more
Labels:
- Labels:
-
Apache Impala