[ANNOUNCE] Cloudera Enterprise new Release, CDH 6.3.1 is Generally Available

[#1]

We are happy to announce Cloudera Enterprise new release, CDH 6.3.1.

What's New?

It is a regular maintenance release, cumulatively includes base 6.3 features, additionally new Dell EMC Isilon support to latest patches and fixes, above existing known limitations.

Cloudera Manager, 6.3.1, the recommended tool for installing Cloudera Enterprise, has its own feature additions, fixes and known limitations updated.

Quick overview of Runtime fixes as follows:

Cloudera Distribution fix/es for 

Kafka

Idempotent and Transactional Capabilities of Kafka that are Incompatible with Sentry

Upstream fix/es for

Apache Avro

HIVE-17829 - Fixed ArrayIndexOutOfBoundsException that occurred when using HBASE-backed tables with Avro schema in Hive2

Apache Hadoop

HADOOP-16018 - DistCp does not reassemble chunks when the value of blocks per chunk is greater than zero.

HDFS

HDFS-12828 - OIV ReverseXML Processor fails with escaped characters.

HDFS-13101 - An fsimage corruption related to snapshots.

HDFS-13709 - Report bad block to NameNode when transfer block encounters EIO exception

HDFS-14148 - HDFS OIV ReverseXML SnapshotSection parser throws exception when there is more than one snapshottable directory.

HDFS-14687 - Standby Namenode does not come out of safemode when EC files are being written.

HDFS-14706 - Checksums are not checked if the block meta file size is less than 7 bytes.

MapReduce 2

MAPREDUCE-7225 - Fix broken current folder expansion during MR job start

YARN

YARN-9667 - Container-executor.c duplicates messages to stdout

YARN-9833 - Race condition when DirectoryCollection.checkDirs() runs during container launch

Apache HBase

HBASE-19893 - restore_snapshot is broken in master branch when region splits

HBASE-20305 - adding options to skip deletes/puts on target when running SyncTable

HBASE-22169 - Open region failed cause memory leak

HBASE-22539 - WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used

HBASE-22617 - Recovered WAL directories not getting cleaned up

HBASE-22690 - Deprecate / Remove OfflineMetaRepair in hbase-2+

HBASE-22759 - Extended grant and revoke audit events with caller info

Apache Hive

HIVE-17829 - Fixed ArrayIndexOutOfBoundsException that occurred when using HBASE-backed tables with Avro schema in Hive2

Hue

HUE-8922 - [frontend] Show dates and times in local format with timezone offset details

HUE-8933 - [editor] Results are not properly cleared in multi-statement execution

HUE-8950 - [core] Saving newly copied Oozie workflow throws an exception

HUE-8979 - [jb] Oozie spark jobs display a NoneType object that is not iterable

Apache Impala

IMPALA-8549 - Added support for scanning DEFLATE text files.

IMPALA-8820 - Fixed an issue where the catalogd process was not found when Impala starts in a cluster.

IMPALA-8847 - The event based automatic metadata invalidation can now correctly ignore empty partition lists generated for certain Hive queries.

Apache Oozie

OOZIE-3397 - Improve logging in NotificationXCommand.

OOZIE-3542 - Handle better HDFS implementations in ECPolicyDisabler.

Apache Sentry

SENTRY-2276 - Sentry-Kafka integration does not support Kafka's Alter/DescribeConfigs and IdempotentWrite operations

SENTRY-2528 - Format exception when fetching a full snapshot

Apache Spark

SPARK-18364 - [YARN] Expose metrics for YarnShuffleService

SPARK-24352 - [CORE][TESTS] De-flake StandaloneDynamicAllocationSuite blacklist test

SPARK-24355 - Spark external shuffle server improvement to better handle block fetch requests.

SPARK-25139 - [SPARK-18406][CORE][2.4] Avoid NonFatals to kill the Executor in PythonRunner

SPARK-25641 - Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100

SPARK-25642 - [YARN] Adding two new metrics to record the number of registered connections as well as the number of active connections to YARN Shuffle Service

SPARK-25692 - [CORE] Remove static initialization of worker eventLoop handling chunk fetch requests within TransportContext. This fixes ChunkFetchIntegrationSuite as well.

SPARK-26615 - [CORE] Fixing transport server/client resource leaks in the core unittests

SPARK-27021 - [CORE] Cleanup of Netty event loop group for shuffle chunk fetch requests

SPARK-28150 - [CORE][FOLLOW-UP] Don't try to log in when impersonating.

SPARK-28150 - [CORE] Log in user before getting delegation tokens.

SPARK-28261 - [CORE] Fix client reuse test

SPARK-28335 - [DSTREAMS][TEST] DirectKafkaStreamSuite wait for Kafka async commit

SPARK-28584 - [CORE] Fix thread safety issue in blacklist timer, tests

How You can get it?

Full software download available at Cloudera Homepage, with support and documentation.

To avoid disrupting currently running CDH workloads, and at Your convenience, You can download the parcel(s) and apply it directly to provisioned clusters via latest Cloudera Manager.

Cloudera archive continues to provide access to Runtime variants, and previous versions.

Thank You for using Cloudera Software!