Member since
10-28-2020
572
Posts
46
Kudos Received
40
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
31 | 02-17-2025 06:54 AM | |
4440 | 07-23-2024 11:49 PM | |
679 | 05-28-2024 11:06 AM | |
1174 | 05-05-2024 01:27 PM | |
748 | 05-05-2024 01:09 PM |
11-21-2022
12:41 PM
@lysConsulting Have you ticked Kudu checkbox under Hive configuration in Cloudera Manager UI? Refer to: https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/kudu_hms.html#concept_enable_hms https://cwiki.apache.org/confluence/display/Hive/Kudu+Integration In CDP : https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/kudu-hms-integration/topics/kudu-hms-enabling.html
... View more
11-15-2022
04:34 AM
@hanumanth Review the logs to get a better idea on what it is failing on. If we have that many partitions, do check what's the value of hive.msck.repair.batch.size and if it is set to 0 at present, you may set this to a value such as 3000 and retry.
... View more
11-11-2022
03:10 AM
Which version are you on? Is this Cloudera Hive distribution? I think UPDATE command is not working because it is not an acid(transactional) table. For update command to work, it has to be an ACID table. @roti
... View more
11-11-2022
02:59 AM
@Shawn Here is a small example on how to find out what's the percentage of NOT NULL value of column maker: select ((tot_cnt - a.null_maker)/tot_cnt)*100 as pcnt_not_null_maker
from
(select count(maker) as null_maker from used_cars where maker is NULL) a
Left outer join
(select count(*) as tot_cnt from used_cars) b You may try this for all individual columns.
... View more
10-18-2022
01:29 PM
@SwaggyPPPP Is this a partitioned table? In that case you could run the ALTER TABLE command as follows: alter table my_table add columns(field4 string,field5 string) CASCADE; Let us know if this issue occurs consistently, after adding new columns, and your Cloudera product version?
... View more
10-18-2022
01:07 PM
1 Kudo
@KPG1 We only support upgrading an existing cluster using Ambari or Cloudera Manager, instead of importing/updating the jars manually. In latest CDP Private cloud base, and our Public Cloud, we are using Hadoop version 3.1.1 at this point.
... View more
10-18-2022
12:56 PM
@ditmarh this might not work in scenarios where the table schema.table is created from Hive, and we are appending to it from Spark. You may try the following command, replacing saveAsTable with insertInto. df.write.mode("append").format("parquet").insertInto("schema.table")
... View more
09-14-2022
01:21 PM
@RamuAnnamalai It looks similar to https://issues.apache.org/jira/browse/IMPALA-10042 Please check what's the value of "Maximum Cached File Handles" under Impala Configuration in CM UI? Set that to Zero(0) and see if the issue still reappears. How do you write to the table? Is there a chance, the data is getting corrupted during the insert?
... View more
09-14-2022
01:07 PM
@Asim- Unless your final table has to be a Hive managed(acid) table then, you could incrementally update the Hive table directly using Sqoop. e.g. sqoop import --connect jdbc:oracle:thin:@xx.xx.xx.xx:1521:ORCL --table EMPLOYEE --username user1 --password welcome1 --incremental lastmodified --merge-key employee_id --check-column emp_timestamp --target-dir /usr/hive/warehouse/external/empdata/ Otherwise, the way you are trying is the actually the way Cloudera recommends it.
... View more
09-05-2022
09:16 AM
@HanzalaShaikh You may consider DLM replication. This is explained here and here. You set the hive.repl.rootdir to set the location where you you want to store the backup, and use the REPL DUMP command to dump your data and metadata: e.g. REPL DUMP db1 WITH('hive.repl.rootdir'='s3a://blah/'); Refer to the Cloudera documentation for for more details and examples.
... View more