Announcing Cloudera Enterprise 6.3.3   We are pleased to announce the release of Cloudera Enterprise 6.3.3, the world’s leading data platform for machine learning and analytics, optimized for the cloud.   This maintenance release includes all the features and capabilities of the previous minor release (Cloudera Enterprise 6.3) with the addition of numerous  important bug fixes.  Also note the current list of known issues with Cloudera Enterprise 6.  We also encourage you to review the new Upgrade Guide that now includes the ability to create customized upgrade instructions based on your specific environment and upgrade path.   New Features & Capabilities   Cloudera Enterprise 6.3.3 now supports running on RedHat Enterprise Linux 7.7 and CentOS 7.7 Cloudera Manager provides new tuning options to increase the scalability of monitoring services for large CDH deployments   Important Note About Installation   Beginning with Cloudera Manager 6.3.3 and CDH 6.3.3, downloading the software will require a valid Cloudera Enterprise license file and/or a username and password obtained from Cloudera.  All Cloudera Manager package, CDH parcel and CDH package repositories now require authentication with valid credentials to access any version labeled 6.3.3 or higher. If you have a valid Cloudera Enterprise or CDP license key, you can generate the download credentials on the CDH download page.  For more information on using these credentials, see the documentation below. Cloudera Installation Guide Cloudera Manager Upgrade Guide CDH Upgrade Guide   Important Note About Cloudera Express   Beginning with release 6.3.3, Cloudera Express has been discontinued and is no longer available. Upgrades to Cloudera Manager 6.3.3 or CDH 6.3.3 and higher are not supported while running Cloudera Express.  A valid Cloudera Enterprise or CDP license must be in place before upgrading to Cloudera Manager 6.3.3 or 7.x, or the upgrade will not be completed. This also applies to trial installations of Cloudera Enterprise and CDP Data Center.   Downgrading from Cloudera Enterprise to Cloudera Express is also no longer supported in Cloudera Manager 6.3.3 and later. An expired Cloudera Enterprise or CDP license, or an expired trial license will disable the Cloudera Manager UI until a valid Cloudera Enterprise or CDP  license key is provided.   As always, we value your feedback and remain committed to Your success! Please provide any comments and suggestions through our Community Forums. ... View more

We are happy to announce Cloudera Enterprise new Runtime release, CDH 6.3.2. What's New? It is a patch release, cumulatively includes base 6.3 features, additional to latest single important fix, above existing known limitations from previous.   Cloudera Manager, 6.3.1, the recommended tool for installing Cloudera Enterprise, is compatible and has no updates this time.   Quick overview of Runtime fixes as follows:   Upstream fix/es for Kudu [KUDU-2990] Kudu cannot distribute libnuma (dependency of memkind). In this release the NVM cache implementation in Kudu has been changed to dynamically link memkind at runtime using dlopen().   How You can get it?   Full software download available at Cloudera Homepage, with support and documentation.   To avoid disrupting currently running CDH workloads, and at Your convenience, You can download the parcel(s) and apply it directly to provisioned clusters via latest Cloudera Manager. Cloudera archive continues to provide access to full Runtime variant. Thank You for using Cloudera Software! ... View more

We are happy to announce Cloudera Enterprise new release, CDH 6.3.1. What's New? It is a regular maintenance release, cumulatively includes base 6.3 features, additionally new Dell EMC Isilon support to latest patches and fixes, above existing known limitations.   Cloudera Manager, 6.3.1, the recommended tool for installing Cloudera Enterprise, has its own feature additions, fixes and known limitations updated.   Quick overview of Runtime fixes as follows:   Cloudera Distribution fix/es for  Kafka Idempotent and Transactional Capabilities of Kafka that are Incompatible with Sentry   Upstream fix/es for Apache Avro HIVE-17829 - Fixed ArrayIndexOutOfBoundsException that occurred when using HBASE-backed tables with Avro schema in Hive2   Apache Hadoop HADOOP-16018 - DistCp does not reassemble chunks when the value of blocks per chunk is greater than zero.   HDFS HDFS-12828 - OIV ReverseXML Processor fails with escaped characters. HDFS-13101 - An fsimage corruption related to snapshots. HDFS-13709 - Report bad block to NameNode when transfer block encounters EIO exception HDFS-14148 - HDFS OIV ReverseXML SnapshotSection parser throws exception when there is more than one snapshottable directory. HDFS-14687 - Standby Namenode does not come out of safemode when EC files are being written. HDFS-14706 - Checksums are not checked if the block meta file size is less than 7 bytes.   MapReduce 2 MAPREDUCE-7225 - Fix broken current folder expansion during MR job start   YARN YARN-9667 - Container-executor.c duplicates messages to stdout YARN-9833 - Race condition when DirectoryCollection.checkDirs() runs during container launch   Apache HBase HBASE-19893 - restore_snapshot is broken in master branch when region splits HBASE-20305 - adding options to skip deletes/puts on target when running SyncTable HBASE-22169 - Open region failed cause memory leak HBASE-22539 - WAL corruption due to early DBBs re-use when Durability.ASYNC_WAL is used HBASE-22617 - Recovered WAL directories not getting cleaned up HBASE-22690 - Deprecate / Remove OfflineMetaRepair in hbase-2+ HBASE-22759 - Extended grant and revoke audit events with caller info   Apache Hive HIVE-17829 - Fixed ArrayIndexOutOfBoundsException that occurred when using HBASE-backed tables with Avro schema in Hive2   Hue HUE-8922 - [frontend] Show dates and times in local format with timezone offset details HUE-8933 - [editor] Results are not properly cleared in multi-statement execution HUE-8950 - [core] Saving newly copied Oozie workflow throws an exception HUE-8979 - [jb] Oozie spark jobs display a NoneType object that is not iterable   Apache Impala IMPALA-8549 - Added support for scanning DEFLATE text files. IMPALA-8820 - Fixed an issue where the catalogd process was not found when Impala starts in a cluster. IMPALA-8847 - The event based automatic metadata invalidation can now correctly ignore empty partition lists generated for certain Hive queries.   Apache Oozie OOZIE-3397 - Improve logging in NotificationXCommand. OOZIE-3542 - Handle better HDFS implementations in ECPolicyDisabler.   Apache Sentry SENTRY-2276 - Sentry-Kafka integration does not support Kafka's Alter/DescribeConfigs and IdempotentWrite operations SENTRY-2528 - Format exception when fetching a full snapshot   Apache Spark SPARK-18364 - [YARN] Expose metrics for YarnShuffleService SPARK-24352 - [CORE][TESTS] De-flake StandaloneDynamicAllocationSuite blacklist test SPARK-24355 - Spark external shuffle server improvement to better handle block fetch requests. SPARK-25139 - [SPARK-18406][CORE][2.4] Avoid NonFatals to kill the Executor in PythonRunner SPARK-25641 - Change the spark.shuffle.server.chunkFetchHandlerThreadsPercent default to 100 SPARK-25642 - [YARN] Adding two new metrics to record the number of registered connections as well as the number of active connections to YARN Shuffle Service SPARK-25692 - [CORE] Remove static initialization of worker eventLoop handling chunk fetch requests within TransportContext. This fixes ChunkFetchIntegrationSuite as well. SPARK-26615 - [CORE] Fixing transport server/client resource leaks in the core unittests SPARK-27021 - [CORE] Cleanup of Netty event loop group for shuffle chunk fetch requests SPARK-28150 - [CORE][FOLLOW-UP] Don't try to log in when impersonating. SPARK-28150 - [CORE] Log in user before getting delegation tokens. SPARK-28261 - [CORE] Fix client reuse test SPARK-28335 - [DSTREAMS][TEST] DirectKafkaStreamSuite wait for Kafka async commit SPARK-28584 - [CORE] Fix thread safety issue in blacklist timer, tests How You can get it?   Full software download available at Cloudera Homepage, with support and documentation.   To avoid disrupting currently running CDH workloads, and at Your convenience, You can download the parcel(s) and apply it directly to provisioned clusters via latest Cloudera Manager. Cloudera archive continues to provide access to Runtime variants, and previous versions. Thank You for using Cloudera Software! ... View more

We are happy to announce Cloudera Enterprise 6.2.1 ! What's New? It is a regular maintenance release, including base release features, additionally latest patches and Upstream fixes packaged.   Apache Hadoop HADOOP-16011 - OsSecureRandom very slow compared to other SecureRandom implementations HADOOP-16018 - DistCp won't reassemble chunks when blocks per chunk > 0. HADOOP-16167 - Fixed Hadoop shell script for Ubuntu 18. HADOOP-16238 - Add the possbility to set SO_REUSEADDR in IPC Server Listener HDFS HDFS-10477 - Stop decommission a rack of DataNodes caused NameNode fail over to standby HDFS-12781 - After Datanode down, In Namenode UI Datanode tab is throwing warning message. HDFS-13101 - Yet another fsimage corruption related to snapshot HDFS-13244 - Add stack, conf, metrics links to utilities dropdown in NN webUI HDFS-13677 - Dynamic refresh Disk configuration results in overwriting VolumeMap HDFS-14111 - hdfsOpenFile on HDFS causes unnecessary IO from file offset 0 HDFS-14314 - fullBlockReportLeaseId should be reset after registering to NN HDFS-14359 - Inherited ACL permissions masked when parent directory does not exist HDFS-14389 - getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured HDFS-14687 - Standby Namenode never come out of safemode when EC files are being written HDFS-14746 - Trivial test code update after HDFS-14687 MapReduce 2 MAPREDUCE-7225 - Fix broken current folder expansion during MR job start YARN YARN-9552 - FairScheduler: NODE_UPDATE can cause NoSuchElementException YARN-9667 - Use setbuf with line buffer to reduce fflush complexity in container-executor. Apache HBase HBASE-19893 - restore_snapshot is broken in master branch when region splits HBASE-21736 - Remove the server from online servers before scheduling SCP for it in hbck HBASE-21800 - RegionServer aborted due to NPE from MetaTableMetrics coprocessor HBASE-21960 - RESTServletContainer not configured for REST Jetty server HBASE-21978 - Should close AsyncRegistry if we fail to get cluster id when creating AsyncConnection HBASE-21991 - Fix MetaMetrics issues - [Race condition, Faulty remove logic], few improvements HBASE-22128 - Move namespace region then master crashed make deadlock HBASE-22144 - Correct MultiRowRangeFilter to work with reverse scans HBASE-22169 - Open region failed cause memory leak HBASE-22200 - WALSplitter.hasRecoveredEdits should use same FS instance from WAL region dir HBASE-22581 - User with "CREATE" permission can grant, but not revoke permissions on created table HBASE-22615 - Make TestChoreService more robust to timing HBASE-22617 - Recovered WAL directories not getting cleaned up HBASE-22690 - Deprecate / Remove OfflineMetaRepair in hbase-2+ HBASE-22759 - Extended grant and revoke audit events with caller info - ADDENDUM Apache Hive HIVE-16811 - Estimate statistics in absence of stats, partially backported, when a query fails with IllegalArgumentException Size requested for unknown type: java.util.Collection HIVE-13278 - Avoid FileNotFoundException when map/reduce.xml is not available Hue HUE-4327 - [editor] Turn off batch mode for query editors HUE-8140 - [editor] Additional improvements to multi statement execution HUE-8691 - [useradmin] Fix group sync fail to import member HUE-8717 - [oozie] Fix Sqoop1 editor fail to execute HUE-8720 - [importer] Fix importer with custom separator HUE-8727 - [frontend] Prevent Chrome from autofilling user name in various input elements HUE-8734 - [editor] Fix zero width column filter in the results HUE-8746 - [pig] Add hcat support in the Pig Editor in Hue HUE-8759 - [importer] Fix import to index, importing to hive instead HUE-8802 - [assist] Fix js exception on assist index refresh HUE-8829 - [core] Fix redirect stops at /hue/accounts/login HUE-8860 - [beeswax] Truncate column size to 5000 if too large HUE-8878 - [oozie] Fix Hive Document Action variable with prefilled value HUE-8879 - [core] Fix ldaptest not allow space in user_filter HUE-8880 - [oozie] Fix KeyError when execute coordinator HUE-8922 - [frontend] Show dates and times in local format with timezone offset details HUE-8933 - [editor] Make sure to clear any previous result when the execute call returns HUE-8950 - [core] Fix error of saving copied document Apache Impala IMPALA-7800 - Impala now times out new connections after it reaches the maximum number of concurrent client connections. The limit is specified by the --fe_service_threads startup flag. The default value is 64 with which 64 queries can run simultaneously. Previously the connection attempts that could not be serviced were hanging infinitely. IMPALA-7802 - Idle client connections are now closed to conserve front-end service threads. IMPALA-8469 - Fixed the issue where Impala clusters with dedicated coordinators incorrectly rejected queries destined for memory pools with configured limits. IMPALA-8549 - Added support for scanning DEFLATE text files. IMPALA-8595 - Impala supports TLS v1.2 with the Python version 2.7.9 and older in impala-shell. IMPALA-8673 - Added the DEFAULT_HINTS_INSERT_STATEMENT query option for setting the default hints for the INSERT statements when no optimizer hint was specified. Apache Kafka KAFKA-7697 - Process DelayedFetch without holding leaderIsrUpdateLock Apache Kudu KUDU-2807 - The system doesn’t crash when a flush or a compaction overlaps with another compaction. KUDU-2871 - (part 1): A temporary fix that pegs maximum TLS version to TLSv1.2. Apache Oozie OOZIE-3365 - Workflow and coordinator action status remain as RUNNING after rerun. OOZIE-3397 - Improve logging in NotificationXCommand. OOZIE-3478 - Oozie needs execute permission on the submitting user's home directory. Apache Sentry SENTRY-2276 - Sentry-Kafka integration does not support Kafka's Alter/DescribeConfigs and IdempotentWrite operations SENTRY-2511 - Debug level logging on HMSPaths significantly affects performance SENTRY-2528 - Format exception when fetching a full snapshot Apache Spark SPARK-25139 - [SPARK-18406][CORE][2.4] Avoid NonFatals to kill the Executor in PythonRunner SPARK-25429 - [SQL] Use Set instead of Array to improve lookup performance SPARK-26003 - Improve SQLAppStatusListener.aggregateMetrics performance SPARK-26089 - [CORE] Handle corruption in large shuffle blocks SPARK-26349 - [PYSPARK] Forbid insecure py4j gateways SPARK-27094 - [YARN] Work around RackResolver swallowing thread interrupt. SPARK-27112 - [CORE] : Create a resource ordering between threads to resolve the deadlocks encountered ... SPARK-28150 - [CORE][FOLLOW-UP] Don't try to log in when impersonating. SPARK-28150 - [CORE] Log in user before getting delegation tokens. SPARK-28335 - [DSTREAMS][TEST] DirectKafkaStreamSuite wait for Kafka async commit Apache Zookeeper ZOOKEEPER-1392 - Request READ or ADMIN permission for getAcl() ZOOKEEPER-2141 - ACL cache in DataTree never removes entries See all fixes in this release, Cloudera Proprietary software fixes included. How You can get it? You can download the parcel(s) from our archive and apply it directly to provisioned clusters without disrupting Your currently running CDH workloads.   Full software download also available at Cloudera Homepage,with support and documentation.   Archive continues to provide access to both variants.   Thank You for using Cloudera Software! ... View more

We are pleased to announce the general availability of Cloudera Enterprise 6.3.0, the world’s leading data platform for machine learning and analytics, optimized for the cloud.    This release delivers a number of new capabilities, improved usability, better performance, and support for more modern Java and identity management infrastructure software.    New capabilities include: Platform highlights: Support for the OpenJDK 11 runtime: all components and tools in Cloudera Enterprise 6 support both the JDK8 and JDK11 Java Virtual Machine (JVM). Updated versions of platform components, including packaging & upgrade support for the following new Apache project versions: Kafka 2.2.1, HBase 2.1.4, Impala 3.2.0 and Kudu 1.10.0. Support for Free IPA and Red Hat IPA Manager: Cloudera Manager & CDH now supports the use of FeeIPA/RedHat IDM as the Kerberos KDC provider for CDH, and for LDAP authentication to Cloudera Enterprise Support for zstd compression with Parquet files, a fast real-time compression algorithm offering an improved speed to compression ratio as well as fast decompression speeds. Both Impala and Spark have been certified with zstd and Parquet. Management highlights: SPNEGO/Kerberos support for Cloudera Manager Admin Console and API, Cloudera Manager now supports strong authentication via Kerberos (using SPNEGO). See Configuring External Authentication and Authorization for Cloudera Manager. TLS certificate expiry monitoring and alerting: Cloudera Manager now alerts you 60 days before the Cloudera Manager Server’s TLS certificate expires, prompting you to rotate (re-generate) the certificates used by Cloudera Manager’s push-button wire encryption system (‘AutoTLS’) Network Performance Inspector now includes a bandwidth test, for verifying sufficient network performance between independent compute and storage clusters Kafka support in Compute Clusters, Independently managed Kafka ‘compute’ clusters can now share a single Sentry in a base CDH cluster for common authorization across all services Governance highlights: Auditing in Virtual Private Clusters. Cloudera Navigator now extracts audit events from all relevant activity within Compute Clusters in addition to collecting audit events upon creation of Compute Clusters and Data Contexts. No lineage or metadata is extracted from services running on Compute clusters. The new behavior is described in detail in Virtual Private Clusters and Cloudera SDX. Search, query, access highlights: Data Cache for Remote Reads (preview feature, off by default): To improve performance on environments with separate storage & compute clusters as well as on object store environments, Impala now caches data for non-local reads (e.g. S3, ABFS, ADLS) on local storage. See Impala Remote Data Cache for the information and steps to enable remote data cache. Automatic Invalidate/Refresh Metadata (preview feature, off by default): When other CDH services update the Hive metastore, Impala users no longer have to issue INVALIDATE/REFRESH in a number of scenarios. See Impala Metadata Management for the information and steps to enable the Zero Touch Metadata feature. Support for Kudu integrated with Hive Metastore, metadata for Kudu tables can now be managed via HMS and shared between Impala and Spark. See Using the Hive Metastore with Kudu for upgrading existing tables. Kudu supports both full and incremental table backups, via a job implemented using Apache Spark. Additionally, it supports restoring tables from full and incremental backups via a restore job implemented using Apache Spark. Kudu can use HDFS, S3 and any Spark-compatible destination to store backups. See the backup documentation for more details. Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with Kerberos by passing negotiation through the HTTP headers. To enable authorization using SPNEGO, set the --webserver_require_spnego command line flag. Query Profile, output enhanced for better monitoring and troubleshooting of query performance. See Impala Query Profile for generating and reading query profile. Security highlights: Kudu now supports native, fine-grained authorization via integration with Apache Sentry, Kudu can now enforce role-based access control policies defined in Sentry. When this feature is turned on, access control is enforced for all clients accessing Kudu, including Impala, Spark and native Kudu clients. See the authorization documentation for more details.   Product software download available, with updated documentation. Please refer to the release notes for a complete list of features, same for base release.   We also encourage you to review the new Upgrade Guide that now includes the ability to create a customized document based on your unique upgrade path.   As always, we love your feedback and remain committed to Your success! Please provide any comments and suggestions through Our Community forums. ... View more

We are happy to announce Cloudera Enterprise 5.16.2.   You can download the parcel(s) and apply it directly to provisioned clusters without disrupting Your currently running CDH workloads.   What's New?   Upstream and other faults fixed: Apache Flume  FLUME-2973 - Deadlock in hdfs sink. FLUME-3223 - Flume HDFS Sink should retry close prior recover lease. Apache Hadoop HADOOP-15442 - ITestS3AMetrics.testMetricsRegister should not know the name of the metrics source. HDFS-11751 - DFSZKFailoverController daemon exits with the wrong status code. HDFS-12683 - DFSZKFailOverController re-order logic for logging exception. HDFS-14111 - hdfsOpenFile on HDFS causes unnecessary IO from file offset 0 MAPREDUCE-6382 - HTML links in the Diagnostics in JHS job overview must not be escaped. MAPREDUCE-7125 - JobResourceUploader creates LocalFileSystem when it's not necessary. MAPREDUCE-7131 - Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate. YARN-4227 - Ignore expired containers from the removed nodes in FairScheduler. YARN-4677 - RMNodeResourceUpdateEvent update from scheduler can lead to race condition. Apache HBase HBASE-16810 - HBase Balancer throws ArrayIndexOutOfBoundsException when regionservers are in /hbase/draining znode and unloaded HBASE-17510 - DefaultMemStore gets the wrong heap size after rollback HBASE-19730 - Backport HBASE-14497 Reverse Scan threw StackOverflow caused by readPt checking HBASE-20604 - ProtobufLogReader#readNext can incorrectly loop to the same position in the stream until the the WAL is rolled HBASE-21275 - Disable TRACE HTTP method for thrift http server HBASE-21546 - ConnectException in TestThriftHttpServer Apache Hive HIVE-12476 - Metastore NPE on Oracle with Direct SQL HIVE-13278 - Avoid FileNotFoundException when map/reduce.xml is not available HIVE-13394 - Analyze table fails in Tez on empty partitions HIVE-13592 - metastore calls map is not thread safe HIVE-14557 - Nullpointer When both SkewJoin and Mapjoin Enabled HIVE-14560 - Support exchange partition between S3 and HDFS tables HIVE-14690 - Query fail when hive.exec.parallel=true, with conflicting session dir HIVE-16839 - Unbalanced calls to openTransaction/commitTransaction when altering the same partition concurrently HIVE-18778 - Needs to capture input/output entities in explain HIVE-20331 - Query with union all, lateral view and Join fails with "cannot find parent in the child operator" HIVE-20678 - HiveHBaseTableOutputFormat should implement HiveOutputFormat to ensure compatibility HIVE-20695 - HoS Query fails with hive.exec.parallel=true HIVE-21028 - get_table_meta should use a fetch plan to avoid race conditions ending up in NucleusObjectNotFoundException HIVE-21044 - Improvments to HMS metrics HIVE-21045 - Add connection pool info and rolling performance info to the metrics system Hue HUE-8388 - [oozie] Make Hue create a new workspace when importing an Oozie workflow instead of using the "deployment_dir" field HUE-8450 - [editor] Embedded mode improvements for previous Hue version HUE-8458 - [frontend] Improve application loading performance HUE-8468 - [frontend] Dynamically adding styles in embedded mode fails in Internet Explorer (throws a Java script exception) HUE-8584 - [useradmin] Errors returned for Add Sync Ldap Group HUE-8585 - [useradmin] Errors returned for Add Sync Ldap Users HUE-8631 - HBase is not accessible by way of the Hue server; instead returns "API Error." HUE-8660 - [core] File browser cannot view files containing a hash (#) in the name HUE-8691 - [useradmin] Attempting to add/sync group will not add users if the objectClass posixGroup exists in the LDAP entry HUE-8692 - [useradmin] Group sync fails if all group members are not found with error "No such object" HUE-8693 - [useradmin] Security application only displays 100 users in the impersonation list HUE-8705 - [oozie] Hidden popup window is blocking the Query drop-down menu and the search box HUE-8709 - [useradmin] Black transparent screen remains after confirmation modal is hidden HUE-8746 - [pig] Add hcat support to the Pig Editor in Hue Apache Impala IMPALA-6323 - Impala now supports a constant in the window specifications. IMPALA-7960 - Impala now returns a correct result when comparing TIMESTAMP to a string literal in a binary predicate where the TIMESTAMP is casted to VARCHAR of smaller length. IMPALA-7961 - Fixed an issue where queries running with the SYNC_DDL query option can fail when the Catalog Server is under a heavy load with concurrent catalog operations of long-running DDLs. IMPALA-8058 - Fixed cardinality estimates for HBase queries, which could sometimes yield hugely high numbers. IMPALA-8109 - Impala can now read the gzip files bigger than 2 GB. IMPALA-8212 - Fixed a race condition in the Kerberos authentication code. Kite KITE-1185 - Make root temp directory path configurable in HiveAbstractDatasetRepository Apache Kudu KUDU-1678 - Fixed a crash caused by a race condition between altering tablet schemas and deleting tablet replicas. KUDU-2195 - Now you can use the ‑‑cmeta_force_fsync flag to fsync Kudu’s consensus metadata more aggressively. Setting this to true may decrease Kudu’s performance, but will improve its durability in the face of power failures and forced shutdowns. The issue was much more likely to happen when Kudu was running on XFS. KUDU-2463 - Fixed an issue in which incorrect results would be occasionally returned in scans following a server restart. CDH-76920 - Fixed the issue where the Kudu CLI crashes when running the 'kudu cluster rebalance' sub-command on some platforms. Apache Oozie OOZIE-3382 - Implement and backportOptimize SshActionExecutor's drainBuffers method Apache Pig PIG-5373 - InterRecordReader might skip records if certain sync markers are used PIG-5374 - Use CircularFifoBuffer in InterRecordReader Apache Sentry SENTRY-2205 - Improve Sentry NN Logging. SENTRY-2301 - Log where sentry stands in the snapshot fetching process, periodically SENTRY-2372 - SentryStore should not implement grantOptionCheck SENTRY-2419 - Log where sentry stands in the process of persisting the snpashot SENTRY-2427 - PortUse Hadoop KerberosName class to derive shortName SENTRY-2428 - Skip null partitions or partitions with null sds entries SENTRY-2437 - PortWhen granting privileges a single transaction per grant causes long delays SENTRY-2490 - PortWhen building a full perm update for each object we only build 1 privilege per role SENTRY-2498 - Exception while deleting paths that does't exist SENTRY-2502 - Modified BackportSentry NN plug-in stops fetching updates from sentry server. SENTRY-2511 - Debug level logging on HMSPaths significantly affects performance Apache Spark SPARK-4224 - [CORE][YARN] Support group acls SPARK-19019 - [PYTHON][BRANCH-1.6] Fix hijacked `collections.namedtuple` and port cloudpickle changes for PySpark to work with Python 3.6.0 Apache Zookeeper  ZOOKEEPER-1392 - Request READ or ADMIN permission for getAcl() ZOOKEEPER-2141 - ACL cache in DataTree never removes entries   See all fixes in this release. See documentation here.   Thank You for using Cloudera Software! ... View more

We are happy to announce CDS 2.4 Release 2 Powered By Apache Spark.   You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads.   This component is generally available and is supported on CDH 5.10 and higher.   What's New in CDS 2.4 release 2? - A new Streamlined API - Performance Improvements - Stream Processing using Dataframes - New machine learning algorithms and model persistence See CDS Powered By Apache Spark Fixed Issues for the list of fixed issues and enhancements.   See documentation and full download available here.   -------------- Want to become a pro Spark user?  Sign up for Apache Spark Training. ... View more

We are happy to announce CDS 2.4 Release 1 Powered By Apache Spark.   You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads.   This component is generally available and is supported on CDH 5.10 and higher.   What's New in CDS 2.4 release 1? - A new Streamlined API - Performance Improvements - Stream Processing using Dataframes - New machine learning algorithms and model persistence See CDS Powered By Apache Spark Fixed Issues for the list of fixed issues.   See documentation and full download available.   Want to become a pro Spark user?  Sign up for Apache Spark Training. ... View more

We are happy to announce CDS 2.1 Release 4 Powered By Apache Spark. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads.   This component is generally available and is supported on CDH 5.7 and higher.   What's New in CDS 2.1 release 4 This is purely a maintenance release. See CDS Powered By Apache Spark Fixed Issues for the list of fixed issues. Download CDS 2.1 Release 4 Powered By Apache Spark. Read the documentation. Want to become a pro Spark user?  Sign up for Apache Spark Training. ... View more