Member since
10-28-2020
389
Posts
18
Kudos Received
23
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
93 | 08-31-2023 03:16 AM | |
165 | 08-21-2023 03:08 AM | |
296 | 08-16-2023 01:06 PM | |
298 | 07-25-2023 07:29 AM | |
386 | 07-03-2023 01:35 PM |
07-03-2023
01:35 PM
@vaibhavgokhale You could try: --conf spark.sql.hive.conf.list="tez.queue.name=queue1"
... View more
06-28-2023
11:31 AM
@Choolake Try this: count1=$(beeline -u "jdbc:hive2://dev-lisa.realm.com:10000/default;principal=hive/dev-lisa.intranet.slt.com.lk@REALM.COM;ssl=true;sslTrustStore=/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks" --showHeader=false --silent=true --outputformat=tsv2 -e 'SELECT count(*) from table_name;')
These beeline flags will remove all the unnecessary texts from the stdout. Compute count2 the same way.
... View more
06-26-2023
06:17 AM
@aafc could you please share the CDP (HIve) version you are using? Also, have you tried with the latest Clouldera JDBC driver for hive?
... View more
06-24-2023
11:53 AM
@rahuledavalath In a 3-node ZK cluster, we can only afford to lose 1 instance/node at once. However, having this cluster across two geo-locations could put us in a difficult position. So, it will be a good idea to build the cluster across 3 regions, with one instance in each.
... View more
06-21-2023
06:39 AM
@Choolake Try : entries=$((count2-count1)) This should work provided we have valid values on both variables.
... View more
06-14-2023
10:47 PM
@xiamu this error could appear if the data nodes are not healthy. Does the job fail repeatedly, or it succeeds at times? Have you tried running it with a different user? This is where it is failing: private void setupPipelineForAppendOrRecovery() throws IOException {
// Check number of datanodes. Note that if there is no healthy datanode,
// this must be internal error because we mark external error in striped
// outputstream only when all the streamers are in the DATA_STREAMING stage
if (nodes == null || nodes.length == 0) {
String msg = "Could not get block locations. " + "Source file \""
+ src + "\" - Aborting..." + this;
LOG.warn(msg);
lastException.set(new IOException(msg));
streamerClosed = true;
return;
}
setupPipelineInternal(nodes, storageTypes, storageIDs);
}
... View more
06-14-2023
10:21 PM
@haihua Do you mean it works from beeline but not from Hive CLI? If it works with beeline, why don't we run it run it with that instead? beeline ... -f query.hql Also, could you try providing required privileges to the role with GRANT OPTION ? Refer to https://docs.cloudera.com/documentation/enterprise/latest/topics/sg_hive_sql.html#grant_privilege_with_grant
... View more
06-09-2023
08:47 AM
@snm1523 From beeline use sys;
select cd_id, count(cd_id) as column_count from columns_v2 group by cd_id order by cd_id asc; -- this will return column_count for each table Every individual table will have a unique cd_id. To map the table names with cd_id, try the following. select t.tbl_name, s.cd_id from tbls t join sds s where t.sd_id=s.sd_id order by s.cd_id asc; You could also merge the two queries to get the o/p together.
... View more
06-09-2023
05:55 AM
@BrianChan Does the AD user have permissions to view the Hive table? This privilege need to be set under Hadoop SQL policies in Ranger. If this is already done, you also need to set required atlas policies. Refer to https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/atlas-securing/topics/atlas-configure-ranger-authorization.html Make sure that the entiity-type is set correctly.
... View more
06-09-2023
05:36 AM
@snm1523 It should be SET FILEFORMAT INPUTFORMAT... Please try it as follows. ALTER TABLE alter_file_format_test SET FILEFORMAT INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe';
... View more