What's New @ Cloudera

We are pleased to announce the release of Impala ODBC v2.5.40 and Impala JDBC v2.5.41.  The release has the following fixes and enhancements: Impala ODBC 2.5.40Enhancements and New Features in Impala ODBC 2.5.40The drivers now supports Impala versions 1.0.1 through 2.9.You can now set configurations options so that they apply to every connection that uses the driver. Before, most configuration options had to be set individually for each connection. The DelegationUIDCase property has been renamed. It is now called DelegationUserIDCase.The driver now automatically canonicalizes the server SPN when using Active Directory Kerberos. Resolved Issues in Impala ODBC 2.5.40On macOS, the driver sometimes terminates unexpectedly when authenticating the connection through Kerberos.Driver installation failing on Linux has been resolved. The issue occurred because the RPM file used an incorrect path for the installation directory. If a query contains a scalar subquery, the driver retu

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.4.0 / CDH 5.12.1. Apache Kudu 1.4.0 / CDH 5.12.1 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.4.0 / CDH 5.12.1, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.4.0 / CDH 5.12.1: KUDU-2085 - Fixed a bug that caused crashes when seeking past the end of prefix-encoded blocks.KUDU-2087 - Fixed an issue where Kudu would fail to start when Kerberos was enabled in FreeIPA-configured deployments.KUDU-2053 - Fixed a race condition in the Java RequestTracker.KUDU-2049 - Fixed an issue where scans on RLE-encoded integer columns would sometimes cause CHECK failures due to the CHECK condition being too strict.KUDU-2052 - Kudu now uses XFS_IOC_UNRESVSP64 ioctl to punch holes on xfs filesystems. This fixes an issue with slow startup times on xfs when hole punching was done via fallocate().KUDU-1956 - Kudu will no longer crash if

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.12.1 (CDH 5.12.1, Cloudera Manager 5.12.1, and Cloudera Navigator 2.11.1). This release fixes key bugs and includes the following: CDH fixes for the following issues:IMPALA-5615 - Compute Incremental stats is broken for general partition expressionsIMPALA-5539 - Reading timestamps from Kudu are wrong with -use_local_tz_for_unix_tsIMPALA-5582 - Sentry privileges assigned to objects defined in uppercase can get deleted from the catalogIMPALA-5500 - Reduce catalog topic size when --compact_catalog_topic is enabledIMPALA-5579 - GetSchemas throws IndexArrayOutOfBoundsException if a table can't be loadedIMPALA-5616 - Add a new flag --enable_minidumpsIMPALA-5524 - NullPointerException during planning with DISABLE_UNSAFE_SPILLS=TRUEHIVE-16845 - INSERT OVERWRITE a table with dynamic partitions on S3 fails with NPEHIVE-13588 - NPE is thrown from MapredLocalTask

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.3.0 / CDH 5.11.2. Apache Kudu 1.3.0 / CDH 5.11.2 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.3.0 / CDH 5.11.2, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.3.0 / CDH 5.11.2:KUDU-2053 - Fixed a race condition in the Java RequestTracker.KUDU-2049 - Fixed an issue where scans on RLE-encoded integer columns would sometimes cause CHECK failures due to the CHECK condition being too strict.KUDU-1963 - Fixed an issue where the Java client misdiagnoses an error and logs a NullPointerException when a connection is closed by the client while a negotiation is in progress.KUDU-1853 - Fixed an issue where data blocks could be orphaned after a failed tablet copy.For a complete list of changes, bug fixes, and known issues, see the Kudu 1.3.0 / CDH 5.11.2 Release Notes. As always, your feedback is appreciated. For general Kudu

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.11.2 (CDH 5.11.2, Cloudera Manager 5.11.2, and Cloudera Navigator 2.10.2). This release fixes key bugs and includes the following: CDH fixes for the following issues:HIVE-11418 - Dropping a database in an encryption zone with CASCADE and trash enabled failsHIVE-11878 - ClassNotFoundException can possibly occur if multiple jars are registered one at a time in HiveHIVE-12762 - Common join on parquet tables returns incorrect result when hive.optimize.index.filter set to trueHIVE-16660 - Not able to add partition for views in hive when sentry is enabledHIVE-16869 - Hive returns wrong result when predicates on non-existing columns are pushed down to Parquet readerIMPALA-5355 - Sentry Privileges and roles updated in the wrong order on impala restartIMPALA-5197 - Parquet scan may incorrectly report "Corrupt Parquet file" in the logsIMPALA-4293 - Query profil

We are pleased to announce the release of Hive ODBC v2.5.24 and Hive JDBC v2.5.19. The release has the following fixes and enhancements: Enhancements and New Features in Hive ODBC 2.5.24Updated configuration option: The name of the Delegation User ID case configuration option has changed from DelegationUIDcase to DelegationUserIDcase. Resolved Issues in Hive ODBC 2.5.24The statement “set hive.support.quoted.identifiers=column” is executed incorrectly.Driver report only read operations are supported in the driver log.The INFINITY value for FLOAT columns are not correctly retrieved.Error/status code returned for query timeout error is incorrect.NaN value retrieved for FLOAT columns is incorrect.The operation handle for metadata API calls is not closed correctly Getting Started with the Cloudera DriverRead the Cloudera ODBC 2.5.24 Driver for Hive Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page. Enhancements and New Features

We are pleased to announce the release of Cloudera Data Science Workbench 1.1.1. This release provides bug fixes and product improvements, as outlined in the Release Notes Keytab AuthenticationWith version 1.1.1, you can now authenticate yourself to CDH cluster by uploading your Kerberos keytab to Cloudera Data Science Workbench. For more information on downloading, installing and using please see the links below:Download Cloudera Data Science WorkbenchCloudera Data Science Workbench OverviewGetting Started with Cloudera Data Science WorkbenchCloudera Data Science Workbench Release NotesInstalling Cloudera Data Science Workbench As always, we welcome your feedback. Please send your comments and suggestions on our community forums.

We are pleased to announce the release of Cloudera Director 2.5.1. Cloudera Director 2.5.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.5.1 release notes. AWS eventual consistency during tagging of instancesWe have recently observed that tagging of instances can take longer than expected in some AWS regions. This can result in timeouts during instance allocation when using Cloudera Director 2.4 and lower. Cloudera recommends upgrading to Cloudera Director 2.5 or higher to avoid this issue, unless you are using AWS GovCloud or China regions. Cloudera Director 2.5 and higher uses the AWS tag-on-create feature when creating instances, resulting in faster instance tagging. The AWS tag-on-create feature is currently not available in AWS GovCloud and China regions. For more info on downloading and installing Director, see the links below.Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorIns

Do you use cloud infrastructure for your CDH clusters? You are invited to share your feedback on your usage and management of cloud infrastructure. You will thereby influence the roadmap of Cloudera Director and other cloud orchestration software from Cloudera. Please share your experiences by taking this short survey:  http://tiny.cloudera.com/director-survey Key Cloudera Director features recently added include:Automatic repair of clustersSupport for long-running clusters: Cloudera Director now syncs with cluster changes made via Cloudera ManagerAWS Spot instance support and improved resiliency to spot instance disappearancesSupport for AWS EBS volumesSupport for Azure ADLS, Azure premium and standard storage For more details on which versions support different features, please refer to the release notes. If you have any questions or comments, you can reach me at vinithra@cloudera.com. Thanks for your input,Vinithra VaradharajanEngineering Manager,

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.9.3 (CDH 5.9.3, Cloudera Manager 5.9.3, and Cloudera Navigator 2.8.3). This release fixes key bugs and includes the following: CDH fixes for the following issues:HIVE-11418 - Dropping a database in an encryption zone with CASCADE and trash enabled failsHIVE-14380 - Queries on tables with remote HDFS paths fail in "encryption" checks.HIVE-14564 - Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsExceptionHIVE-16660 - Not able to add partition for views in hive when sentry is enabledIMPALA-5355 - Sentry Privileges and roles updated in the wrong order on impala restartIMPALA-5193 - Impala reads gzip compressed text as binary when skip.header.line.count > 0IMPALA-5197 - Parquet scan may incorrectly report "Corrupt Parquet file" in the logsIMPALA-4293 - Query profile should include error logFor a full list

We are pleased to announce Release 1.1 of Cloudera Data Science Workbench. Cloudera Data Science Workbench (CDSW) allows data scientists to use their favorite open source libraries and languages — including R, Python, and Scala — directly with existing secure CDH clusters, accelerating analytics projects from exploration to production and bringing self-service data science to the enterprise. Enhancements in this release include:GPU Support: Cloudera Data Science Workbench now enables popular deep learning frameworks to run on GPUs, both on-premises and in the cloud.Embedded Web UIs: Users can work with the Apache Spark Web UI for Spark sessions. Other interactive web applications like TensorBoard, Shiny, and Plotly now appear directly in the workbench.Enhanced Job Scheduling: Cloudera Data Science Workbench users can now schedule jobs directly from external schedulers or orchestration systems via the new Jobs API.CDSW Release 1.1 also includes many other enhancements to

We are happy to announce Apache Spark 2.2 release 1. You can download the parcel and apply it directly to provisioned clusters without disrupting your currently running Spark workloads. This component is generally available and is supported on CDH 5.8 through CDH 5.12. What's New in Cloudera Distribution of Apache Spark 2.2 Release 1Support for CDH 5.12 and associated features.Support for using Spark 2 jobs to read and write data on the Azure Data Lake Store (ADLS) cloud service.Cloudera Distribution of Apache Spark 2.2 requires JDK 8. Issues Fixed in Cloudera Distribution of Apache Spark 2.2 release 1[SPARK-10364][SQL] Support Parquet logical type TIMESTAMP_MILLIS[SPARK-10849][SQL] Adds option to the JDBC data source write for user to specify database column type for the create table[SPARK-12868][SQL] Allow adding jars from HDFS[SPARK-14503][ML] spark.ml API for FPGrowth[SPARK-16101][HOTFIX] Fix the build with Scala 2.10 by explicit typed argument[SPARK-16122][CORE] Add

We are pleased to announce the release of Cloudera Director 2.5.0. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. What's New in Cloudera Director 2.5.0 Automatic repair: Cloudera Director can now automatically repair instances that get terminated unexpectedly for clusters that have auto-repair enabled. Significant improvement in resilience to unexpected instance terminations during bootstrap and grow operations. This improves support for AWS spot instances, as well as resilience of clusters using on-demand instances.Cloudera Director can now configure and create external accounts in Cloudera Manager that include cloud provider login credentials in order to manage the use of additional cloud provider functionality, including object stores, such as Amazon

Cloudera is pleased to announce that Cloudera Enterprise 5.12 is now generally available (GA). The release includes enhancements for running in cloud environments (with broader ADLS support and improved AWS Spot Instance support), usability and productivity improvements for both data science and analytic workloads, as well as performance gains and self-service performance management across a range of workloads.  As usual, there are also a number of quality enhancements, bug fixes, and other improvements across the stack. Here is a partial list of what’s included (see the Release Notes for a full list): Core PlatformImproved AWS Spot Instance Support: Cloudera Director 2.5 makes using AWS Spot Instance much easier and more reliable. Director can now recover from Spot instances disappearing during initial cluster spin up, grow operations, and in steady state. In addition, Cloudera Manager can be made aware of Spot instances for improved job reliability.OpenStack Reference Archi

Hello CDH Users, We are pleased to announce the release of Impala ODBC v2.5.39. The release has the following fixes and enhancements: Enhancements & New Features:Restrict catalog functions to current schema. If a schema is not specified for a catalog function, or a schema is specified with the wildcard character %, you can choose to restrict the catalog function to the current schema. This may improve performance. To do this, select the Restrict Metadata with Current Schema GUI option, or set the Current Schema Restricted Metadata connection setting to 1.  Resolved IssuesThe driver incorrectly adds a length to all the type name in CAST statements during query translation.In the driver log, the driver is reported as read-only, despite being read/write enabled.The driver does not correctly retrieve infinity value for FLOAT columns.The driver does not correctly retrieve NaN (Not a Number) value for FLOAT columns. Getting Started with the Cloudera DriverRead the Clo

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.2.0- CDH 5.10.2. Apache Kudu 1.2.0- CDH 5.10.2 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.2.0- CDH 5.10.2, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.2.0- CDH 5.10.2:KUDU-1933 - Fixed an issue that truncated the 64-bit log index in the OpId to 32 bits, causing overflow of the log index.KUDU-1607 - Fixed an issue where Kudu could not delete failed tablets using the DROP TABLE command.KUDU-1905 - Allow reinserts on tables when all columns are part of the primary key.KUDU-1893 - Made a fix to avoid incorrect NULL results and ensure evaluation of predicates for columns added after table creation.For a complete list of changes, bug fixes, and known issues, see the Kudu 1.2.0/ CDH 5.10.2 Release Notes. As always, your feedback is appreciated. For general Kudu questions, visit the community page. If you have any qu

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.10.2 (CDH 5.10.2, Cloudera Manager 5.10.2, and Cloudera Navigator 2.9.2). This release fixes key bugs and includes the following: CDH fixes for the following issues:HADOOP-12751 - While using kerberos Hadoop incorrectly assumes names with '@' to be non-simpleHBASE-16630 - Fragmentation in long running Bucket CacheHIVE-12768 - Thread safety: binary sortable serde decimal deserializationHIVE-14564 - Column Pruning generates out of order columns in SelectOperator which cause ArrayIndexOutOfBoundsExceptionIMPALA-4725 - Query option to control Parquet array resolutionIMPALA-4546 - Fix Moscow timezone conversion after 2014IMPALA-4822 - Implement dynamic log level changesIMPALA-5487 - Race in runtime-profile.cc::toThrift() can lead to corrupt profiles being generated while query is runningFor a full list of upstream JIRAs fixed in CDH 5.10.2, see the issues

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.11.1 (CDH 5.11.1, Cloudera Manager 5.11.1, and Cloudera Navigator 2.10.1). This release fixes key bugs and includes the following: CDH fixes for the following issues:HADOOP-14028 - S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failuresHIVE-12768 - Thread safety: binary sortable serde decimal deserializationIMPALA-5251 - DecimalAvgFinalize() gets the wrong arg typeIMPALA-5186 - Handle failed CreateAndOpenScanner() in MT scan.IMPALA-3641 - DROP / CREATE sequence on same table failed with "table already exists".IMPALA-4902 - Concurrent DDL may fail with a ConcurrentModificationException.IMPALA-5253 - Use appropriate transport for StatestoreSubscriberIMPALA-5145 - CTAS failing when creating from a view with error "Unsupported type 'null_type'"For a full list of upstream JIRAs fixed in CDH 5.11.1, see the

Hello CDH Users, We are pleased to announce the release of Impala JDBC v2.5.38. The release has the following fixes: Resolved IssuesDriver incorrectly passes SSLTrustStore configuration to the server as server-side property. Translation fails due to the driver treating coercion between STRING and BIGINT columns as an invalid operation.When certain fetch sizes are set for the result set, the driver incorrectly returns an error while executing a query.  Getting Started with the Cloudera DriverRead the Cloudera JDBC 2.5.38 Driver for Impala Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page. As always, we welcome your feedback. Please send your comments and suggestions to the user group of through our community forums. You can also file bugs through our external Jira projects issues.cloudera.org.

Cloudera is happy to announce the availability of parcels and packages for Apache Kudu 1.3.0- CDH 5.11.1. Apache Kudu 1.3.0- CDH 5.11.1 is a bug fix release, with no new features or backwards incompatible changes. To upgrade Kudu to 1.3.0- CDH 5.11.1, see Upgrade Parcels or Upgrade Packages. Issues Fixed in Kudu 1.3.0- CDH 5.11.1 KUDU-1999 - Fixed an issue where the Kudu Spark connector would fail to kinit with the principal and keytab provided to a job.KUDU-1993 - Fixed a validation issue with grouped gflags.KUDU-1981 - Fixed an issue where Kudu server components would fail to start on machines with fully-qualified domain names longer than 64 characters when security was enabled. This was due to hard-coded restrictions in the OpenSSL library.KUDU-1607 - Fixed a case in which a tablet replica on a tablet server could retain blocks of data which prevented it from being fully deleted.KUDU-1933 - Fixed an issue in which a tablet server would crash and fail to restart after

We are pleased to announce the release of Cloudera Director 2.4.1. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise.  Cloudera Director 2.4.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.4.1 release notes. Download Cloudera DirectorCloudera Director Release NotesGetting Started with Cloudera DirectorInstalling Cloudera Director ClientInstalling Cloudera Director Server As always, we are happy to hear your feedback. Please post your questions and comments on our community forum.

Dear CDH, Cloudera Manager, and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.8.5 (CDH 5.8.5, Cloudera Manager 5.8.5, and Cloudera Navigator 2.7.5.). This release fixes key bugs and includes the following: CDH fixes for the following issues:FLUME-2798 - Malformed Syslog messages can lead to OutOfMemoryExceptionFLUME-2812 - Fix semaphore leak causing java.lang.Error: Maximum permit count exceeded in MemoryChannelHADOOP-13579 - Fix source-level compatibility after HADOOP-11252HDFS-11229 - HDFS-11056 failed to close meta fileHIVE-11487 - Add getNumPartitionsByFilter api in metastore apiHIVE-12465 - Hive might produce wrong results when (outer) joins are mergedHIVE-12768 - Thread safety: binary sortable serde decimal deserializationHUE-3065 - [oozie] Sub-workflow submitted from coordinator gets parent workflow graphHUE-4147 - [useradmin] Ignore (objectclass=*) filter when searching for LDAP usersIMPALA-3018 - Don't return NULL on z

Dear Cloudera User,   We are excited to announce the General Availability (GA) release of Cloudera Altus.  Cloudera Altus is a platform-as-a-service offering that makes it dramatically easier and more cost-effective to run Cloudera in the cloud. The initial service - available today - targets foundational data engineering workloads and will allow you to quickly and easily spin up Cloudera clusters and run your data pipelines.   Why Cloudera Altus: The platform-as-a-service deployment model eliminates the complexity and overhead associated with cloud and cluster operations for the end user. Instead, users can quickly launch data engineering jobs on-demand and focus on their workloads rather than cluster operations. Altus leverages the trusted, enterprise-grade Cloudera distribution, allowing easy migration of on-prem workloads to and from the cloud with minimal risk. Altus provides interoperable workflows between the data engineering offering and Cloudera’s machine learn

Hello CDH Users, We are pleased to announce the release of Hive ODBC v2.5.22 driver. This release has the following fixes and enhancements: Enhancements & New FeaturesUpdated third-party library dependencies The driver has been updated to incorporate newer versions of the third-party librarydependencies. Resolved IssuesWhen attempting to use the Windows trust store on Windows Server 2016, anaccess violation exception occursDriver converts COALESCE function to less efficient CASE statementConfiguration tool for DSN-less connections, the EnableAuto Reconnect option is disabled by defaultIn some cases, when upgrading the macOS driver from an earlier version to a later one, the installation process fails to update the odbcinst.ini configuration fileSegmentation fault in Driver Manager detection on LinuxIf you use an alias to refer to the Hive server host name while authenticating throughKerberos, the driver fails to connectGetting Started with the Cloudera DriverRead the

Cloudera is pleased to announce the General Availability (GA) of Cloudera Data Science Workbench 1.0. Cloudera Data Science Workbench enables self-service data science and engineering for your secure CDH environment.   Why Cloudera Data Science Workbench: Provide Data Scientists the freedom to use R, Python, or Scala with their favorite libraries and frameworks Directly access data in secure, compliant Hadoop clusters through Apache Spark and Apache Impala Research in a reproducible and collaborative way Run on-premises or in the cloud For more details, see the Cloudera Data Science Workbench release notes.   You can download the Cloudera Data Science Workbench here or register for the final part of our webinar series here. Installation and other documentation for Cloudera Data Science Workbench can be found here.   As always, we are happy to hear your feedback. Please post your comments and suggestions in our community forums.