Member since
10-28-2020
304
Posts
14
Kudos Received
13
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
110 | 10-18-2022 01:07 PM | |
261 | 09-05-2022 09:16 AM | |
412 | 08-18-2022 06:56 AM | |
634 | 08-05-2022 02:12 AM | |
679 | 08-03-2022 06:15 AM |
01-10-2023
09:53 PM
@anjel Thank you! I can see that Support has reached out to the ODBC driver team. We'll wait for their response.
... View more
01-06-2023
03:46 AM
@anjel Are we saying we are able to see the column comments but table comment missing? Could you please try adding table comment as follows, instead of setting it in table properties? CREATE TABLE name(column1 string comment 'column comment') COMMENT "table comment")
... View more
01-03-2023
04:39 AM
@anjel Power BI does not have a native Hive connector. I guess you could only connect using an ODBC connector. I just tried listing a table using DbVisualizer, and and it shows the column comments as REMARKS. So, I would say it is specific to the Hive client(Power BI)
... View more
01-02-2023
12:35 AM
1 Kudo
@jass @OmerSl Try disabling hive.server2.tez.initialize.default.sessions. Go to CM UI > Hive on Tez > Configuration > HiveServer2 Advanced Configuration Snippet (Safety Valve) for hive-site.xml Add: Name: hive.server2.tez.initialize.default.sessions Value: false Save and restart the service. This should help resolve the issue. More information on this property can be found here : https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/configuring-apache-hive/topics/hive-on-tez-configurations.html
... View more
01-02-2023
12:01 AM
@saicharan This fails with the authentication error. HTTP Response Code : 401 Verify the authentication details you are passing in the Adapter configuration
... View more
12-18-2022
11:22 PM
@anjel could you share the query you are running from Hue or powerbi that you expect to return column/table comments? Does describe <table> not return the column comments in powerBI? What's the JDBC/ODBC driver version you are using and is it shared by Cloudera?
... View more
12-01-2022
01:01 AM
@aleezeh I guess you could reach out to Trino support for their inputs. But it looks like we are missing a dependent elasticsearch jar in the classpath.
... View more
11-30-2022
03:49 AM
@aleezeh We seem to be missing the jar for the class org.elasticsearch.ElasticsearchStatusException Please make sure that you have elasticsearch-7.6.0.jar is present in the classpath. This jar comes bundled in ranger-${RANGER_VERSION}-admin.tar.gz.
... View more
11-30-2022
03:37 AM
@d_liu It seems like it was using a cookie based authentication( hive.server2.thrift.http.cookie.auth.enabled=true set in HS2 config), and it failed due to some error at the server end. Do you have multiple Hive servers configured with Hue, and now it is trying to connect to a different instance? As suggested by @Basim earlier, restart HS2 and Hue service, and retry.
... View more
11-21-2022
12:41 PM
@lysConsulting Have you ticked Kudu checkbox under Hive configuration in Cloudera Manager UI? Refer to: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kudu_hms.html#concept_enable_hms https://cwiki.apache.org/confluence/display/Hive/Kudu+Integration In CDP : https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/kudu-hms-integration/topics/kudu-hms-enabling.html
... View more
11-15-2022
04:34 AM
@hanumanth Review the logs to get a better idea on what it is failing on. If we have that many partitions, do check what's the value of hive.msck.repair.batch.size and if it is set to 0 at present, you may set this to a value such as 3000 and retry.
... View more
11-11-2022
03:10 AM
Which version are you on? Is this Cloudera Hive distribution? I think UPDATE command is not working because it is not an acid(transactional) table. For update command to work, it has to be an ACID table. @roti
... View more
11-11-2022
02:59 AM
@Shawn Here is a small example on how to find out what's the percentage of NOT NULL value of column maker: select ((tot_cnt - a.null_maker)/tot_cnt)*100 as pcnt_not_null_maker
from
(select count(maker) as null_maker from used_cars where maker is NULL) a
Left outer join
(select count(*) as tot_cnt from used_cars) b You may try this for all individual columns.
... View more
10-18-2022
01:29 PM
@SwaggyPPPP Is this a partitioned table? In that case you could run the ALTER TABLE command as follows: alter table my_table add columns(field4 string,field5 string) CASCADE; Let us know if this issue occurs consistently, after adding new columns, and your Cloudera product version?
... View more
10-18-2022
01:07 PM
1 Kudo
@KPG1 We only support upgrading an existing cluster using Ambari or Cloudera Manager, instead of importing/updating the jars manually. In latest CDP Private cloud base, and our Public Cloud, we are using Hadoop version 3.1.1 at this point.
... View more
10-18-2022
12:56 PM
@ditmarh this might not work in scenarios where the table schema.table is created from Hive, and we are appending to it from Spark. You may try the following command, replacing saveAsTable with insertInto. df.write.mode("append").format("parquet").insertInto("schema.table")
... View more
09-14-2022
01:21 PM
@RamuAnnamalai It looks similar to https://issues.apache.org/jira/browse/IMPALA-10042 Please check what's the value of " Maximum Cached File Handles" under Impala Configuration in CM UI? Set that to Zero(0) and see if the issue still reappears. How do you write to the table? Is there a chance, the data is getting corrupted during the insert?
... View more
09-14-2022
01:07 PM
@Asim- Unless your final table has to be a Hive managed(acid) table then, you could incrementally update the Hive table directly using Sqoop. e.g. sqoop import --connect jdbc:oracle:thin:@xx.xx.xx.xx:1521:ORCL --table EMPLOYEE --username user1 --password welcome1 --incremental lastmodified --merge-key employee_id --check-column emp_timestamp --target-dir /usr/hive/warehouse/external/empdata/ Otherwise, the way you are trying is the actually the way Cloudera recommends it.
... View more
09-05-2022
09:16 AM
@HanzalaShaikh You may consider DLM replication. This is explained here and here. You set the hive.repl.rootdir to set the location where you you want to store the backup, and use the REPL DUMP command to dump your data and metadata: e.g. REPL DUMP db1 WITH('hive.repl.rootdir'='s3a://blah/'); Refer to the Cloudera documentation for for more details and examples.
... View more
08-31-2022
05:05 AM
@mohammad_shamim Did you have Hive HA configured in CDH cluster, in that case, you need to make sure that there are equal number of HS2 instances created in the CDP cluster, because without that HA cannot be attained. Also, make sure that there is no Hiveserver2 instance created under "Hive" service in CDP. It should only be present under Hive on Tez service.
... View more
08-18-2022
06:56 AM
@ssuja I am afraid it's not achievable using Ranger. If you already have a data directory owned by a specific user, say user1, you may create a policy in Ranger providing hive and other users access to that directory path(URI), and keep the physical path owned by user1 itself. See, if this is something you can work with. I should also mention, creating an external Hive table without Location clause, will create a directory with hive ownership, for Impersonation is disabled in Hive.
... View more
08-12-2022
11:27 AM
Hi @ssuja there is a Hive property that would help you achieve what you are aiming for. Look for hive.server2.enable.doAs under Hive on Tez configurations and enable it. However, there is a catch. This property needs to be disabled if you are using Ranger for authorization. If you are not using Ranger, and using Storage Based Authorization(which is not the recommended in CDP), then you could definitely enable this. Refer to the doc here.
... View more
08-05-2022
02:12 AM
1 Kudo
@xinghx The only difference between CDP 7.1.1 and 7.1.7 is HIVE-24920. In your test case, the CREATE TABLE statement is creating an External table with " TRANSLATED_TO_EXTERNAL" table property set to "TRUE". Your second query to change the table to a Managed/acid table does not really work, so that query has no impact apart from just adding a table property. Now coming to the RENAME query, I notice it does not change the location in CDP 7.1.1 either. Please refer to the attachment. In CDP 7.1.7(SP1) it does change the location if we have "TRANSLATED_TO_EXTERNAL" = "TRUE", If we set it to false, we have the same behavior as 7.1.1. alter table alter_test set tblproperties("TRANSLATED_TO_EXTERNAL"="FALSE"); I hope this helps.
... View more
08-03-2022
06:15 AM
1 Kudo
@xinghx This is an expected behavior in later version of CDP. Please refer to this Release note. If yours is a managed table, in the default warehouse location, the HDFS path will be renamed, the way you expect it to. However, if you plan to rename an External table, you will also need to change the location accordingly: ALTER TABLE <tableName> RENAME TO <newTableName>;
ALTER TABLE <newTableName> set location "hdfs://<location>";
... View more
08-01-2022
01:36 PM
1 Kudo
@Imran_chaush If you are on CDP, and using Ranger for authorization, then you may check the audit log to see which users tried to access that specific database and table. Else, you will have to read the raw log file to see what are the queries run on a specific table, and then try to find out the users submitting those queries. e.g. grep -E 'Compiling.*<table name>' /var/log/hive/hadoop-cmf-hive_on_tez-HIVESERVER2-node1.log.out Column 5 is your session ID, and you may grep for the session ID again to find the user associated with it.
... View more
08-01-2022
12:40 PM
@Caliber The following command should work: # for hql in {a.hql,b.hql}; do beeline -n hive -p password --showheader=false --silent=true -f $hql; done
... View more
04-01-2022
10:41 AM
@mattyseltz the select * query is probably submitting a plain fetch task , which does not involve running any Tez task in yarn containers. The Tez errors in my understanding must be independent of ODBC driver. You may attach your error log here, or create a support case.
... View more
03-29-2022
12:02 PM
@mattyseltz what's the ODBC version you are using, and also could you share HDP/CDP version? It is possible that the said version of ODBC driver does not support the Hive version in use. Where did you download the 32-bit ODBC driver? If you do not see any detailed error, have you tried enabling DEBUG logging in ODBC driver, and see if that gives you more info?
... View more
03-24-2022
12:16 PM
2 Kudos
@Ging I don't think there is much in the liquibase extension that could impose security risks. But, it's better to check with Liquibase. About Hive and Impala JDBC drivers, you could download the latest from Cloudera website, and not 2.6.4/2.6.2 as mentioned in the Liquibase blog. Very soon we are going to release newer versions that address the recent log4j vulnerabilities.
... View more
12-16-2021
12:23 PM
@Gcima009 are you trying to collect the logs with the same user that you submitted the job with? This query completed the map phase, and failed in reducer phase. If you are not able to collect the app logs, do check the HS2 log with the query ID hive_20211210173528_ff76c3df-a33b-41d0-b328-460c9b65deda if you get more information what caused the job to fail.
... View more