What's New @ Cloudera

We are pleased to announce the general availability of the Cloudera Connector Powered by Teradata 1.6c5.  New Features in Cloudera Connector Powered by Teradata Version 1.6c5Upgrades the JDBC driver to version 15.10.00.22 and the TDCH library to version 1.5.0. These libraries contain several bug fixes and improvements.Adds the --schema argument, used to override the <td-instance> value in the connection string of the Sqoop command. For example, if the connection string in the Sqoop command is jdbc:teradata:// <td-host>/DATABASE=database1, but you specify --schema database2, your data is imported from database2 and not database1. If the connection string does not contain the DATABASE parameter — for example jdbc:teradata://<td-host>/CHARSET=UTF8) — you can also use the --schema databasename argument to have Sqoop behave as if you specified the jdbc:teradata://<td-host>/DATABASE=databasename,CHARSET=UTF8 connection string. For more details on n

Cloudera is pleased to announce a new minor release of our supported packaging of Apache Accumulo for use on CDH 5.This release adds support for:Kerberos authentication for Accumulo ClientsRHEL 7 / CentOS 7 supportRunning Accumulo on top of Apache HDFS Transparent EncryptionAccumuloStorgeHandler for Apache Hive access to data stored in AccumuloApache Spark processing of data stored in Accumulo through AccumuloInputFormat and AccumuloOutputFormatUsers are cautioned to read the installation guide for known issues and unsupported upstream features.Existing users should see a new parcel available for Accumulo 1.7.2-cdh5.5.0.This release is supported with Cloudera Manager and CDH versions 5.5.0 or later. Cloudera recommends using the latest version of Cloudera Manager available.Docs: http://tiny.cloudera.com/accumulo-docsCloudera Manager 5.8.2 Download: http://tiny.cloudera.com/cm-5.8.2Accumulo 1.7.2 on CDH 5 (CDH 5.5.0 or later): http://archive.cloudera.com/accumulo-c5/List of JIRAs includ

We are pleased to announce the release of Cloudera Enterprise 5.7.4 (CDH 5.7.4, Cloudera Manager 5.7.4, Cloudera Navigator 2.6.4, and Key Trustee KMS 5.7.4). This release fixes key bugs and includes the following:CDH fixes for the following issues:HADOOP-8436 - NPE In getLocalPathForWrite ( path, conf ) when the required context item is not configuredHDFS-8269 - getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atimeHDFS-9781 - FsDatasetImpl#getBlockReports can occasionally throw NullPointerExceptionMAPREDUCE-6628 - Potential memory leak in CryptoOutputStreamHBASE-16284 - Unauthorized client can shutdown the clusterHIVE-14436 - Hive 1.2.1/Hitting "ql.Driver: FAILED: IllegalArgumentException Error"HIVE-14743 - ArrayIndexOutOfBoundsException - HBASE-backed views query with JOINsIMPALA-3682 - Do not retry unrecoverable socket creation errorsIMPALA-4020 - Handle external conflicting changes to HMS gracefullySPARK-8428 -Fix integer o

We are pleased to announce the release of Impala JDBC v2.5.34 driver. This release has the following fixes and enhancements: Enhancements and new features:Authentication method and transport protocol. You can now specify if SASL should be used in conjunction with the 'Username and Password' Authentication Mechanism (AuthMech=3) by using the Use Sasl connection property. Previously this was done through the transport Mode connection property, which has now been removed. See Release Notes for JDBC for details on all of the fixes. Getting Started with the Cloudera DriversRead the Cloudera JDBC 2.5.34 Driver for Impala Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page. As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org.

Dear CDH, Cloudera Manager, and Cloudera Navigator users,We are pleased to announce the release of Cloudera Enterprise 5.8.2 (CDH 5.8.2, Cloudera Manager 5.8.2, and Cloudera Navigator 2.7.2). This release fixes key bugs and includes the following:CDH fixes for the following issues:HADOOP-8436 - NPE In getLocalPathForWrite ( path, conf ) when the required context item is not configuredHBASE-15496 - Throw RowTooBigException only for user scan/getHDFS-9137 - DeadLock between DataNode#refreshVolumes and BPOfferService#registrationSucceededHDFS-10525 - Fix NPE in CacheReplicationMonitor#rescanCachedBlockMapHIVE-14006 - Hive query with UNION ALL fails with ArrayIndexOutOfBoundsExceptionIMPALA-3646 - Handle corrupt RLE literal or repeat counts of 0IMPALA-3820 - Handle linkage errors while loading Java UDFs in CatalogSPARK-16106 - [CORE] TaskSchedulerImpl should properly track executors added to existing hostsSQOOP-2999 - Sqoop ClassNotFoundException (org.apache.commons.lang3.StringUtils) is t

Cloudera is happy to announce the availability of parcels and packages for Kudu 1.0.0. After approximately a year of beta releases, Apache Kudu has reached version 1.0.0. This version number signifies that the development team feels that Kudu is stable enough for usage in production environments.This is the first non-beta release of the Apache Kudu project. (Although because Kudu is not currently integrated into CDH, it is not yet an officially supported CDH component.)Kudu 1.0.0 delivers a number of new features, bug fixes, and optimizations. We are also releasing a refresh of the Impala Kudu parcel.To upgrade Kudu to 1.0.0, see Upgrade Parcels or Upgrade Packages.In addition, you can read more about Kudu 1.0.0 in the vision blog post we just published. New Features in Kudu 1.0.0Removal of multiversion concurrency control (MVCC) history is now supported. This is known as tablet history GC. This allows Kudu to reclaim disk space, where previously Kudu would keep a full history of

We are pleased to announce the release of CDH 5.4.11. This release fixes key bugs and includes the following: HADOOP-8436 - NPE In getLocalPathForWrite ( path, conf ) when the required context item is not configuredHADOOP-10048 - LocalDirAllocator should avoid holding locks while accessing the filesystemHADOOP-11901 - BytesWritable fails to support 2G chunks due to integer overflowHBASE-14359 - HTable#close will hang forever if unchecked error/exception thrown in AsyncProcess#sendMultiActionHBASE-16207 - can't restore snapshot without "Admin" permissionHDFS-4660 - Block corruption can happen during pipeline recoveryHDFS-10360 - DataNode might format directory and lose blocks if current/VERSION is missing.HIVE-13251 - Hive can't read the decimal in AVRO file generated from previous versionIMPALA-2711 - Fix memory leak in Rand().YARN-5048 - DelegationTokenRenewer#skipTokenRenewal might throw NPEFor a full list of upstream JIRAs fixed in CDH 5.4.11, see the issues fixed section of th

We are pleased to announce the release of CDH 5.5.5. This release fixes key bugs and includes the following: FLUME-2918 - Speed up TaildirSource on directories with many filesHADOOP-12963 - Allow using path style addressing for accessing the s3 endpointHBASE-15811 - Batch Get after batch Put does not fetch all CellsHDFS-9805 - Add server-side configuration for enabling TCP_NODELAY for DataTransferProtocol and default it to trueHDFS-10396 - Using -diff option with DistCp may get "Comparison method violates its general contract" exceptionHIVE-11827 - STORED AS AVRO fails SELECT COUNT(*) when emptyHIVE-13527 - Using deprecated APIs in HBase client causes ZooKeeper connection leaksHUE-3866 - [core] Hue CPU reaches ~100% usage while uploading files with SSL to HTTPFS/WebHDFSMAPREDUCE-6473 - Job submission can take a long time during Cluster initializationOOZIE-2345 - Parallel job submission for forked actionsSENTRY-1265 - Sentry service should not require a TGT as it is not talking to

We are pleased to announce the release of CDH 5.7.3 This release fixes key bugs and includes the following:CDH fixes for the following issues:FLUME-2821 - KafkaSourceUtil Can Log Passwords at Info remove logging of security related data in older releases.HADOOP-11361 - Fix a race condition in MetricsSourceAdapter.updateJmxCache.HDFS-8581 - ContentSummary on / skips further counts on yielding lock.MAPREDUCE-6675 - TestJobImpl.testUnusableNode failed.YARN-5048 - DelegationTokenRenewer#skipTokenRenewal may throw NPE.HBASE-14818 - user_permission does not list namespace permissions.HIVE-11980 - Follow up on HIVE-11696, exception is thrown from CTAS from the table with table-level serde is Parquet while partition-level serde is JSON.HIVE-13749 - Memory leak in Hive Metastore.HUE-4477 - [security] Select All is not filtering out the non visible roles from the selection .IMPALA-3441 - , IMPALA-3659: check for malformed Avro data.SENTRY-1252 - grantServerPrivilege and revokeServerPrivilege sho

We are pleased to announce the release of Cloudera Director 2.1.1. Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 2.1.1 provides bug fixes and and product improvements, as outlined in the Cloudera Director 2.1.1 release notes. Cloudera Director Release NotesDownload Cloudera DirectorGetting Started with Cloudera DirectorInstalling Cloudera Director ClientInstalling Cloudera Director Server As always, we are happy to hear your feedback. Please post your questions and comments on our community forum.

Hello CDH Users,We are pleased to announce the release of Hive ODBC v2.5.20 driver. This release has the following fixes and enhancements:Enhancements & New FeaturesDelegate Kerberos credentials. You can now have the driver forward your Kerberos user credentials to the server to simplify the authentication process.Optimized Fast SQLPrepare behavior. The Fast SQLPrepare driver configuration option (the FastSQLPrepare key) is now disabled for non-SELECT queries. This ensures that the driver retrieves the necessary result set metadata at prepare time.Resolved IssuesUnicode characters in parameter values causing errors. Returning errors for some queries with dates in them. Unable to create new tables when Unicode character types option set.Getting Started with the Cloudera DriverRead the Cloudera ODBC 2.5.20 Driver for Hive Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page.As always, we welcome your feedback. Please send your comments and sug

Dear CDH, Cloudera Manager, and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.7.2 (CDH 5.7.2, Cloudera Manager 5.7.2, and Cloudera Navigator 2.6.2).   This release fixes key bugs and includes the following: CDH fixes for the following issues: HADOOP-11409 - FileContext.getFileContext can stack overflow if default fs misconfigured HADOOP-12787 - KMS SPNEGO sequence does not work with WebHDFS HADOOP-13251 - Authenticate with Kerberos credentials when renewing KMS delegation token HDFS-10360 - DataNode may format directory and lose blocks if current/VERSION is missing HBASE-11625 - Reading data block throws "Invalid HFile block magic" and can not switch to hdfs checksum HIVE-9499 - hive.limit.query.max.table.partition makes queries fail on non-partitioned tables HIVE-10685 - Alter table concatenate operator will cause duplicate data IMPALA-1928 - Fix Thrift client transport wrapping order HUE-4113 - [Pig] Hue bre

We are pleased to announce the release of the Cloudera Distribution of Apache Kafka 2.0.2 for CDH 5.  Apache Kafka is a highly scalable, distributed, publish-subscribe messaging system. Apache Kafka 2.0.2 is maintenance release for Kafka 2.0. Notable Issues Fixed in Cloudera Distribution of Apache Kafka 2.0.2: KAFKA-3495: NetworkClient.blocking Send And Receive should rely on requestTimeout.KAFKA-2998: Log warnings when client is disconnected from bootstrap brokers.KAFKA-3488: Avoid failing of unsent requests in consumer where possible.KAFKA-3528: Handle wakeups while rebalancing more gracefully.KAFKA-3594: After calling MemoryRecords.close() method, hasRoomFor() method should return false.KAFKA-3602: Rename RecordAccumulator dequeFor() and ensure proper usage.KAFKA-3789: Upgrade Snappy to fix Snappy decompression errors.KAFKA-3830: getTGT() debug logging exposes confidential information.KAFKA-3840: Allow clients default OS buffer sizes.KAFKA-3691: Confusing logging during metadat

Cloudera is excited to announce the general availability of Cloudera Enterprise 5.8! Main highlights of this release include Impala read/write support on Amazon S3, a redesigned SQL query editor GUI, the expansion of role-based access control functionality to Cloudera Search, and the GA of Cloudera Navigator Optimizer to facilitate and optimize workload migrations.   For those new to it, Cloudera Navigator Optimizer (previously in beta) is a cloud-based service that helps with offload planning and active data optimization for Apache Hadoop. For example, it provides workload visibility and assessments for building a predictable offload plan, adapting to evolving Hadoop data and workload demands, and optimizing query performance in Impala and Apache Hive. (For more details about Cloudera Navigator Optimizer, check out the release blog post.)   As usual, Cloudera Enterprise 5.8 contains a long list of quality enhancements and bug fixes (learn more about our multi-dimensiona

We are pleased to announce the release of Cloudera ODBC 2.5.34 Driver for Impala. This release has the following fixes and enhancements:  Enhancements and New FeaturesSupport for queries that use the EXISTS syntax.Additional SQL Column optimization for catalog functions that include schema restrictions. Resolved IssuesQuery fails if the column name contains a character from the Latin-1 Supplement character set.Warning dialog when non-Admin user clicks OK on the DSN dialog should be suppressed.Driver reports statement length limit as 16 KB. Getting Started with the Cloudera DriversRead the Cloudera ODBC 2.5.34 Driver for Impala Release Notes and Installation Guide.Download the connector from the Cloudera Connectors page. As always, we welcome your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external JIRA projects on issues.cloudera.org. 

We are pleased to announce the release of Cloudera Director 2.1.0.Cloudera Director provides a simple, reliable, enterprise-grade way to deploy, scale, and manage Apache Hadoop in the cloud of your choice. Cloudera Director is available as a free download for use with Cloudera Express and is fully supported with Cloudera Enterprise. Cloudera Director 2.1.0 includes the following new features: Usage-based billing for Cloudera servicesSupport for running Cloudera Manager and CDH clusters on Microsoft AzureSupport for deploying Cloudera Manager and CDH on a different cloud provider or region than Cloudera Director with a simple network setupSupports Cloudera 5.7 out of the boxVarious bug fixes and improvements Links to the product and documentation are listed below:Download Cloudera Director from: http://www.cloudera.com/content/support/en/downloads.htmlView the documentation:Cloudera Director Release NotesGetting StartedInstalling Cloudera Director ClientInstalling Cloudera Director

 We are pleased to announce the release of Impala JDBC v2.5.32 driver. This release has the following fixes and enhancements: Enhancements and new features:Support added for optimizing driver performance for INSERT statements.You can use the new Optimized Insert connection property to optimize driver performance when executing INSERT statements. This optimization is enabled by default.Support added for configuring the Thrift transport protocol.You can use the new transport Mode connection property to specify the transport protocol that the driver uses in the Thrift layer. The default AuthMech value now depends on the transport Mode setting. For more information, see the Cloudera JDBC Driver for Impala Installation and Configuration Guide.Implemented server version checking for the getTypeInfo() method.When you call the getTypeInfo() method, the driver now checks the version of the Impala server that you are connecting to and only returns data types that are supported by that Impal

Dear CDH, Cloudera Manager, and Cloudera Navigator users,We are pleased to announce the release of Cloudera Enterprise 5.7.1 (CDH 5.7.1, Cloudera Manager 5.7.1, and Cloudera Navigator 2.6.1).  This release fixes key bugs and includes the following:CDH fixes for the following issues:HBASE-15234 - The ReplicationCleaner process can abort if its connection to ZooKeeper is inconsistentHDFS-8211 - DataNode UUID is always null in the JMX counterYARN-4414 - NodeManager connection errors are retried at multiple levelsYARN-4629 - Distributed shell breaks under strong securityHIVE-13401 - Kerberized HS2 with LDAP auth enabled fails Kerberos/delegation token authenticationHIVE-13530 - Hive on Spark throws Kryo exception in some casesIMPALA-3133 - Wrong privileges after a REVOKE ALL ON SERVER statementIMPALA-3139 - Fix drop table statement to not drop views and vice versaSENTRY-1190 - IMPORT TABLE silently fails if Sentry is enabledFor a full list of upstream JIRAs fixed in CDH 5.7.1, see the

We are pleased to announce the release of Cloudera Enterprise 5.5.4 (CDH 5.5.4, Cloudera Manager 5.5.4, and Cloudera Navigator 2.4.4).  This release fixes key bugs and includes the following:CDH fixes for the following issues:FLUME-2886 - Optional Channels can cause OOMsHADOOP-12718 - Incorrect error message by fs -put local dir without permissionHDFS-9406 - FSImage may get corrupted after deleting snapshotHDFS-9514 - TestDistributedFileSystem.testDFSClientPeerWriteTimeout failing; exception being swallowedHDFS-9701 - DN may deadlock when hot-swapping under loadHBASE-15234 The Replication Cleaner process can abort if its connection to ZooKeeper is inconsistentMAPREDUCE-6460 - TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException failsMAPREDUCE-6528 - Memory leak for HistoryFileManager.getJobSummary()YARN-3446 - FairScheduler headroom calculation should exclude nodes in the blacklistHBASE-15019 - Replication stuck when HDFS is restartedHIVE-10303 - HIVE-9471 bro

We are pleased to announce the release of Cloudera Enterprise 5.6.1 (CDH 5.6.1, Cloudera Manager 5.6.1, and Cloudera Navigator 2.5.1).  This release fixes key bugs and includes the following:CDH fixes for the following issues:FSImage may get corrupted after deleting snapshotThe ReplicationCleaner process can abort if its connection to ZooKeeper is inconsistentFLUME-2886 - Optional Channels can cause OOMsHADOOP-12604 - Exception may be swallowed in KMSClientProviderHADOOP-12718 - Incorrect error message by fs -put local dir without permissionMAPREDUCE-6528 - Memory leak for HistoryFileManager.getJobSummary()HIVE-12790 - Metastore connection leaks in HiveServer2IMPALA-2711 - Fix memory leak in Rand()IMPALA-2820 - Support unquoted keywords as struct-field namesSENTRY-835 - Drop table leaves a connection open when using metastorelistenerFor a full list of upstream JIRAs fixed in CDH 5.6.1, see the issues fixed section of the Release Notes.Cloudera Manager fixes for the following issue

If you happen to be in the Austin area, you should consider participating in our Hackathon this coming Sunday May 15.  Hosted at the Cloudera office and sponsored by Cloudera Cares.   The hackathon will focus on reducing mosquito borne virus infections by analyzing data on water, mapping mosquito travel, and historical virus analysis.   515 Congress Ave., Suite 1212, Austin, TX 10:30am to 8:00pm   RSVP and further details here.

We are pleased to announce the release of Hive ODBC v2.5.19 and Impala ODBC v2.5.33 drivers. This release has the following fixes and enhancements:   Hive ODBC v2.5.19Enhancements and New Features-Support added for Red Hat Enterprise Linux (RHEL) 7 and CentOS 7-Updated authentication support. For consistency with Microsoft Azure HDInsight updates, the Windows Azure -HDInsight Emulator Authentication mechanism (AuthMech=5) is now deprecated. The driver no longer supports connections to Hive server instances that run on Windows Azure HDInsight Emulator.   Resolved Issues-When attempting to connect to the server using a non-Windows version of the driver with Service Discovery Mode enabled, the client stops working.-When executing a parameterized INSERT statement on a DATE, DECIMAL, or TIMESTAMP column, an error occurs.-When executing a query that contains a large number of filters using OR operators, an error occurs.-When executing a query that uses regular expressions in the fi

We are pleased to announce the release of Cloudera Enterprise 5.3.10 (CDH 5.3.10, Cloudera Manager 5.3.10, and Cloudera Navigator 2.2.10).  This release fixes key bugs and includes the following:CDH fixes for the following issues:HDFS-9406 - When deleting a snapshot that contains the last record of a given INode, the fsimage may become corrupt because the create list of the snapshot diff in the previous snapshot and the child list of the parent INodeDirectory are not cleaned.HADOOP-12464 - Interrupted client may try to fail over and retryHADOOP-12604 - Exception may be swallowed in KMSClientProviderHDFS-8211 - DataNode UUID is always null in the JMX counterHDFS-9092 - NFS silently drops overlapping write requests and causes data copying to failHDFS-9358 - TestNodeCount#testNodeCount timed outMAPREDUCE-6302 - Incorrect headroom can lead to a deadlock between map and reduce allocationsYARN-3464 - Race condition in LocalizerRunner kills localizer before localizing all resourcesYARN-4

We are pleased to announce the release of Impala JDBC v2.5.31 driver. This release has the following fixes and enhancements: Issues resolved:When translating a query that contains an implicit join, the driver disables the implicit join by using CROSS JOIN syntax.If an invalid server-side property is specified, the driver fails to connect. The driver now posts a warning for the invalid server-side property and continues the connection process.When handling multiple connections at the same time, in some cases the driver returns a conversion error when attempting to convert INT data into Long data.The driver no longer removes hints from queries. To preserve hints, the driver does not translate queries that contain hints even when it is configured to work in translation mode (UseNativeQuery=0).Driver returns error when casting data to type TIMESTAMP.When the Driver.getPropertyInfo() method is called, the driver returns a null pointer exception.Driver casts NULL values to type STRING regard

All,   Cloudera's docs have a new look and improved performance & usability. Read the details here:   http://blog.cloudera.com/blog/2016/04/check-out-those-new-and-improved-cloudera-docs/ We look forward to your feedback!