Member since
10-28-2020
571
Posts
46
Kudos Received
39
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
4432 | 07-23-2024 11:49 PM | |
671 | 05-28-2024 11:06 AM | |
1172 | 05-05-2024 01:27 PM | |
741 | 05-05-2024 01:09 PM | |
802 | 03-28-2024 09:51 AM |
12-15-2021
10:02 AM
@Gcima009 It will be difficult to say what's the issue is from this error stack. Could you generate yarn application log for this job, and review it or attach it here, or only paste the detailed error message? yarn logs -applicationId application_1639152705224_0018 > app_log.out
... View more
11-18-2021
12:13 PM
@hxn Please locate java.security file and use "jdk.tls.disabledAlgorithms" to disable TLSv1, TLSv1.1. If you upgrade Java, you will have to redo it. e.g. # find /usr/java/jdk1.8.0_232-cloudera/ -iname java.security
/usr/java/jdk1.8.0_232-cloudera/jre/lib/security/java.security
# grep -i jdk.tls.disabledAlgorithms /usr/java/jdk1.8.0_232-cloudera/jre/lib/security/java.security
# jdk.tls.disabledAlgorithms=MD5, SSLv3, DSA, RSA keySize < 2048
jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, MD5withRSA, DH keySize < 768, 3DES_EDE_CBC
# certificates such as jdk.tls.disabledAlgorithms or
... View more
11-16-2021
10:10 AM
Hi @Korez Then, pease consider setting the properties that I mentioned earlier. set hive.server2.tez.sessions.per.default.queue=3 //Number of AM containers/queue
set hive.server2.tez.initialize.default.sessions=true
set hive.prewarm.enabled=true
set hive.prewarm.numcontainers=2
set tez.am.container.reuse.enabled=true
set tez.am.container.idle.release-timeout-max.millis=20000
set tez.am.container.idle.release-timeout-min.millis=10000 This will help keep AM containers up and ready for a hive query.
... View more
11-14-2021
02:25 AM
Hi @Korez, There are a bunch of Hive properties that we can make use of to make sure there are a number of AM containers that keep running for the default queues. hive.server2.tez.default.queues=queue1,queue2,queue3
set hive.server2.tez.sessions.per.default.queue=3 //Number of AM containers/queue
set hive.server2.tez.initialize.default.sessions=true
set hive.prewarm.enabled=true
set hive.prewarm.numcontainers=2
set tez.am.container.reuse.enabled=true
set tez.am.container.idle.release-timeout-max.millis=20000
set tez.am.container.idle.release-timeout-min.millis=10000 When you submit the job, do you specify the Tez queue name(tez.queue.name) explicitly? If you do, in that case, they might not use the existing AM containers from the pool, but create a new one. 443 seconds is still very long time to launch a container. Do check if you have any resource constraints in YARN.
... View more
11-14-2021
01:59 AM
@mhchethan Yes, you could mention multiple LDAP URLs separated by spaces. Hive will try the URLs in the mentioned order until a connection is successful. Ref: hive.server2.authentication.ldap.url
... View more
11-14-2021
12:48 AM
1 Kudo
@HareshAmin As you correctly said, Impala does not support the mentioned OpenCSVSerde serde. So, you could recreate the table using CTAS, with a storage format that is supported by both Hive and Impala. CREATE TABLE new_table
STORED AS PARQUET
AS
SELECT * FROM aggregate_test;
... View more
10-25-2021
01:31 PM
1 Kudo
@hxn Could you enter the password here instead of the the path to keystore pass file? > SSLKeyStorePwd=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-host_key.pw
... View more
10-16-2021
04:19 AM
@lucid will the column changes be consistent across all the tables? We could try "select * from <tbl1>", however it would require the schema of the table same in all the tables, else it will return semantic exception.
... View more
10-15-2021
02:46 PM
@lucid Do you mean, the position of the columns or the names of the columns change, or the data? As long as we are aware of the column names(data types should be consistent as well), we should be good to write our UNION statement. Please clarify if my understanding is wrong. As you want to write the physical data into a table, you could consider using Hive Materialised view. If your tables tab1, tab2 are ACID(transactional) tables, every time the data changes in those tables through any INSERT operation, the MV data also get refreshed.
... View more
10-13-2021
11:18 AM
@Glive I couldn't test it right now, but could you try the following? beeline -u "jdbc:hive2://server.com:10200/default;user=bob;password='is your uncle'" Within single quotes try to escape white space with a back slash(\), if it does not work. Else, let beeline prompt you for the password: beeline -u "jdbc:hive2://server.com:10200/default" -n bob -p When it prompts you, enter the password. This was introduced in HIVE-13589
... View more