@nikkie_thomas You can set below if you are using Tez   set hive.merge.mapfiles=true; set hive.merge.mapredfiles=true; set hive.merge.smallfiles.avgsize=<some value>; set hive.merge.size.per.task=<some value>; set hive.merge.tezfiles=true; ... View more

@luigui @MBohlmann  I believe you are hitting https://issues.apache.org/jira/browse/HIVE-22260 ... View more

@sokrates, Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.  ... View more

@enirys  You could refer to the following doc for Hive tuning: https://docs.cloudera.com/HDPDocuments/HDP2/HDP-2.6.5/bk_hive-performance-tuning/content/ch_connectivity-admission-control.html#guidelines-hiveserver2-heaps   If you have other services running on the same HS2 node as well, you might want to reduce Hive heap size  or move a service to a different node. Are you expecting too many connections as explained in the above doc, else you might want to bring down HS2 heap size? If you do not see too many connections, but notice high heap usage, you might want to take a heap dump as @asish mentioned, and see if there is memory leak.   Load balancing across HS2 could take place based on how you are accessing Hive. You could use zooKeeper based connection string. ... View more

I tried both principal=hive/_HOST and principal=hive-myCluster/_HOST and in both cases the krb5kdc.log doesn't mention any incorrect zookeeper/server.com@REALM.COM incorrect name but still outputs the same error to the command line: ``` Error: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper (state=,code=0) ``` ... View more

yes that was indeed a problem 🙂   I was about to comment on that ... View more

@BigData-suk    When you say it is hung at reducer level,so all the containers take more time or few containers in the reudcer takes lot of time and hung. There is a data skewness,if few conatimers at reducer level takes time.   You have to re-write the query if there is a data skweness ... View more

@Mas_Jamie apologies. Unfortunately we dont have the document 😞 ... View more

Hi @asish ,   I have uploaded the Nifi flow definition, so you can reproduce my issue.(In txt file because I can't upload json extension, just change the extension)   It generates a csv:   ``` allstring,mixedtype "foo","bar" "foo2",2 "foo3","bar2" ```   Then convert it to Parquet (V1). The issue comes from UpdateHive processor : Error while compiling statement: FAILED: Execution Error, return code 40000 from org.apache.hadoop.hive.ql.ddl.DDLTask. java.lang.UnsupportedOperationException: Unknown field type: uniontype<int,string>   The issue is not present in Nifi when I convert to another format (Avro for example). But is present when I tried to read back the data from the table (from Hue): AnalysisException: Could not load table open_data.test from catalog CAUSED BY: TableLoadingException: Could not load table open_data.test from catalog CAUSED BY: TException: TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL, error_msgs:[TableLoadingException: Unsupported type 'uniontype' in column 'mixedtype' of table 'test']), lookup_status:OK)     Currently, I am on the last version of Nifi CFM in CDP: 7.2.11 - Flow Management Light Duty with Apache NiFi, Apache NiFi Registry   Thank you for your help,   Best regards, ... View more