@aafc could you please share the CDP (HIve) version you are using? Also, have you tried with the latest Clouldera JDBC driver for hive?  ... View more

@rahuledavalath In a 3-node ZK cluster, we can only afford to lose 1 instance/node at once. However, having this cluster across two geo-locations could put us in a difficult position. So, it will be a good idea to build the cluster across 3 regions, with one instance in each. ... View more

@Choolake Try : entries=$((count2-count1)) This should work provided we have valid values on both variables. ... View more

@xiamu this error could appear if the data nodes are not healthy. Does the job fail repeatedly, or it succeeds at times? Have you tried running it with a different user?  This is where it is failing:   private void setupPipelineForAppendOrRecovery() throws IOException { // Check number of datanodes. Note that if there is no healthy datanode, // this must be internal error because we mark external error in striped // outputstream only when all the streamers are in the DATA_STREAMING stage if (nodes == null || nodes.length == 0) { String msg = "Could not get block locations. " + "Source file \"" + src + "\" - Aborting..." + this; LOG.warn(msg); lastException.set(new IOException(msg)); streamerClosed = true; return; } setupPipelineInternal(nodes, storageTypes, storageIDs); }   ... View more

@haihua Do you mean it works from beeline but not from Hive CLI? If it works with beeline, why don't we run it run it with that instead? beeline ... -f query.hql Also, could you try providing required privileges to the role with GRANT OPTION ?  Refer to https://docs.cloudera.com/documentation/enterprise/latest/topics/sg_hive_sql.html#grant_privilege_with_grant ... View more

@BrianChan Does the AD user have permissions to view the Hive table? This privilege need to be set under Hadoop SQL policies in Ranger. If this is already done, you also need to set required atlas policies.  Refer to https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/atlas-securing/topics/atlas-configure-ranger-authorization.html Make sure that the entiity-type is set correctly. ... View more

@mwblee I am not sure if you are using any Cloudera Hive distribution, if you are, consider upgrading to latest CDP version where we have fixed many issues around compactor initiator/worker/cleaner.  e.g. For initiator:  upstream Jiras - HIVE-21917, HIVE-22568, HIVE-22081   For this specific issue, you may check take a look at multiple factors, such as Hive metastore being overloaded, slow/large(certain txn related tables) metastore database. You may enable DEBUG logging in Hive metastore, and this will provide more information on why/where the compactor is stuck. If you are using opensource Hive, upgrade to Hive 4.x; you will have much better experience w.r.t. compaction. ... View more