Announcements

Dear CDH, Cloudera Manager, Impala and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.2.1 (CDH 5.2.1, Cloudera Manager 5.2.1, and Cloudera Navigator 2.1.1)   This release is focused on fixing key bugs and includes the following:   CDH Fixes Oozie: Using cron-like syntax for Coordinator frequencies could result in duplicate actions in certain cases; this is now fixed (OOZIE-2063) YARN: Handle app-recovery failures gracefully (YARN-2010) Impala: Memory leak with string functions (IMPALA-1397) Impala: IllegalStateException when inserting results of a window function (IMPALA-1400) Impala: Read errors with Parquet files (IMPALA-1401) Impala: Regex functions don’t accept shorthand such as \d (IMPALA-1410) Impala: Queries fail with metastore exception after upgrade and compute stats (IMPALA-1416) Impala: Crashes due to bug in ClientCacheHelper (IMPALA-1445) Cloudera Manager Fixed metric collection for CDH 5.

We're pleased to announce the release of Cloudera Enterprise 5.2 (comprising CDH 5.2, Cloudera Manager 5.2, Cloudera Director 1.0, and Cloudera Navigator 2.1).   This release reflects our continuing investments in Cloudera Enterprise's main focus areas, including security, integration with the partner ecosystem, and support for the latest innovations in the open source platform (including Impala 2.0, its most significant release yet, and Apache Hive 0.13.1). It also includes a new product, Cloudera Director, that streamlines deployment and management of enterprise-grade Hadoop clusters in cloud environments; new component releases for building real-time applications; and new support for significant partner technologies like EMC Isilon. Furthermore, this release ships the first results of joint engineering with Intel, including WITH GRANT OPTION for Hive and Impala and performance optimizations for MapReduce.   Here are some of the highlights (incomplete; see the respective Re

We're pleased to announce the release of Kite SDK 0.17.0. This release updates the examples to CDH 5, defaults Parquet to the non-durable mode from 0.14 and prior, adds support for namespaces, and adds a kite-minicluster for easier development and integration testing against single-node Hadoop deployments. For more details see the release notes and the documentation.

Hello CDH and Impala Users,   We are pleased to announce the release of version 2.5.12 of the ODBC Driver for Apache Hive and version 2.5.20 of the ODBC driver for Impala.  These versions contain bug fixes including one that affected the decimal data type. These drivers work for previous versions of HiveServer2 in CDH 4.1 or higher and Impala 1.0 or higher.   Getting started with the Cloudera ODBC Drivers:   Read the Cloudera ODBC 2.5 Driver for Impala release notes and installation guide Read the Cloudera ODBC 2.5 Driver for Apache Hive release notes and  installation guide Download the connector from the Cloudera Connectors page As always, we are happy to hear your feedback. Please send your comments and suggestions to cdh-user@cloudera.org or post to our new Community Forums.   Kind regards, The Cloudera Team

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.1.3.   Cloudera Enterprise 5.1.3 This release is focused on fixing key bugs and includes the following. CDH Fixes HADOOP-11035 - distcp on mr1(branch-1) fails with NPE using a short relative source path. HBASE-11349 - [Thrift] support authentication/impersonation HBASE-11446 - Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder HBASE-11457 - Increment HFile block encoding IVs accounting for ciper's internal use HBASE-11474 - [Thrift2] support authentication/impersonation HBASE-11565 - Stale connection could stay for a while HBASE-11627 - RegionSplitter's rollingSplit terminated with "/ by zero", and the _balancedSplit file was not deleted properly HBASE-11788 - hbase is not deleting the cell when a Put with a KeyValue, KeyValue.Type.Delete is submitted HBASE-11828 - callers of SeverName.valueOf should use equals and not == HDFS-4257 - The ReplaceData

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.1.2 (CDH 5.1.2, Cloudera Manager 5.1.2, Cloudera Navigator 2.0.1) and CDH 5.0.4.   Cloudera Enterprise 5.1.2 This release is focused on fixing key bugs and includes the following.   CDH Fixes FLUME-2438 - Make Syslog source message body configurable HBASE-11052 - Sending random data crashes thrift service HBASE-11143 - Improve replication metrics HBASE-11609 - LoadIncrementalHFiles fails if the namespace is specified HDFS-6114 - Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr HDFS-6640 - [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).). HDFS-6703 - NFS: Files can be deleted from a read-only mount HDFS-6788 - Improve synchronization in BPOfferService with read write lock HDFS-6825 - Edit log corruption due to delayed block removal HUE-221

We're pleased to announce the release of Kite SDK 0.16.0.This release adds support for Apache Spark, adds a CLI transform command for dataset-to-dataset ETL, and adds a CDH 5 parent POM for building Kite applications targeting CDH 5.   For more details see the release notes and the documentation.

Dear Cloudera Manager users,   We are pleased to announce the release of Cloudera Manager 5.1.1 and 4.8.4.  These are maintenance releases and have a number of bug fixes.   Cloudera Manager 4.8.4 Bug Fixes:   Directory quota in Cloudera Manager shows incorrect values. Activity Monitor in Cloudera Manager doesn't request new http service ticket. Many Hive replications running at the same time can cause some of them to fail with "Read timed out" errors. Multiple Hive replications running at the same time can cause some of them to fail with "command succeeded, but unable to fetch results" errors. This has been noted to occur with as few as 20 replications. Multiple Hive replications started at exactly the same time (typically triggered by replication schedules) can cause one or more of them to complete successfully without actually replicating the desired metadata. Replications can be affected by other replications or commands running at the same time.&nb

Cloudera is pleased to announce support for Accumulo 1.6.0 on both CDH 4 and CDH 5.   As of version 5.1.0, Cloudera Manager now ships with support to deploy and manage Accumulo 1.6.0 on CDH 4.6.0 or later and CDH 5.1.0 or later.   * Docs: http://tiny.cloudera.com/accumulo-docs * CM 5.1.0 Download: http://tiny.cloudera.com/cm-5.1.0 * Accumulo 1.6.0 on CDH 5 (CDH5.1.0 or later): http://archive.cloudera.com/accumulo-c5/ * Accumulo 1.6.0 on CDH 4 (CDH4.6.0 or later): http://archive.cloudera.com/accumulo/

Dear CDH, Cloudera Manager, Impala and Search Users,     We are thrilled to announce the GA release of Cloudera Enterprise 5.1 (CDH 5.1 and Cloudera Manager 5.1),  Cloudera Navigator 2.0, Impala 1.4 for CDH 4, Impala ODBC v2.5.17 and Hive ODBC v2.5.10.   Cloudera Enterprise 5.1 Cloudera Enterprise 5.1 contains a number of new features and component versions including the ones below: Apache Hadoop HDFS Extended ACL Support NFS Enhancements HTTPS support YARN YARN + Impala GA: CDH 5.1 now supports three modes of Resource Management: Static Partitioning (CDH 4.3+) Admission Control (CDH 5.0+) Dynamic Prioritization (CDH 5.1+) As part of Dynamic Prioritization, Llama now supports HA Configuration similar to other HA elements (Active-Passive) Apache HBase HBase 0.98.1 rebase Improved WAL write performance Reverse scans MapReduce over snapshots Apache Spark Rebase to Spark 1.0 Spark Streaming integration with Kerber

Version 0.15.0 contains the following notable changes:   Kite artifacts are built against Apache Hadoop 2 and related projects, and are now available in Maven Central. Added new introduction and concepts documentation. Added a new Datasets convenience class for opening and working with Datasets, superseding DatasetRepositories. Deprecated partition related methods in Dataset in favor of the views API. Added a CLI copy task for copying datasets and also for dataset format conversion and data compaction. Added an application parent POM that makes it easy to use Kite in a Maven project. The examples now use this parent POM. Updated to Crunch 0.10.0 Morphlines Library Added morphline command that parses an InputStream that contains protobuf data: readProtobuf (Rober Fiser via whoschek) Added morphline command that extracts specific values from a protobuf object, akin to a simple form of XPath: extractProtobufPath

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.0.3 (CDH 5.0.3 and Cloudera Manager 5.0.2).   This release is focused on fixing key bugs and includes the following: CDH Fixes FLUME-2245  - HDFS files with errors unable to close FLUME-2416 - Use CodecPool in compressed stream to prevent leak of direct buffers HBASE-10871  Indefinite OPEN/CLOSE wait on busy RegionServers HDFS-5891 - WebHDFS should not try connecting the DataNode during redirection HDFS-6021 - NPE in FSImageFormatProtobuf upgrading from layout -52 to -53 HDFS-6077 - running slive with WebHDFS on secure HA cluster fails with unknown host exception HDFS-6340 - DataNode can't finalize upgrade HDFS-6475 - WebHDFS clients fail without retry because incorrect handling of StandbyException HDFS-6510 - WebHDFS clients clear the delegation token on retry (for HA), thus failing retry requests HDFS-6527 - Edit log corruption due to defer

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.0.2 (CDH 5.0.2 and Cloudera Manager 5.0.2). This release is focused on fixing key bugs and includes the following:   Cloudera Manager Cloudera Manager Impala Query Monitoring works with Impala 1.3.1 CDH Fixes HADOOP-10556: Add toLowerCase support to auth_to_local rules for service name HADOOP-10638: Updating hadoop-daemon.sh to work as expected when nfs is started as a privileged user. HADOOP-10639: FileBasedKeyStoresFactory initialization is not using default for SSL_REQUIRE_CLIENT_CERT_KEY HADOOP-10658: SSLFactory expects truststores being configured HBASE-6990: Pretty print TTL HBASE-10312: Flooding the cluster with administrative actions leads to collapse HBASE-10371: Compaction creates empty hfile, then selects this file for compaction and creates empty hfile and over again HDFS-6326: WebHdfs ACL compatibility is broken HIVE-6913: Hive u

Dear CDH, Cloudera Manager, and Search Users,   We are pleased to announce the release of CDH 4.7 and Search 1.3.  These are both minor releases that contain multiple bug fixes.   CDH 4.7 Bug fixes in CDH 4.7 include: HBase HBASE-10514  - Forward port HBASE-10466, possible data loss when failed flushes HBASE-10257 - Master aborts due to assignment race HBASE-8912 - AssignmentManager throws IllegalStateException from PENDING_OPEN to OFFLINE HDFS HDFS-6289 - HA failover can fail if there are pending DataNode messages for DataNodes which no longer exist HDFS-5944 - LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ right and cause SecondaryNameNode failed do checkpoint HDFS-6191 - Disable quota checks when replaying edit log HDFS-4943 - WebHdfsFileSystem does not work when original file path has encoded chars HDFS-5064 - Standby checkpoints should not block concurrent readers HDFS-6160 - TestSafeMode occasionally fails HDFS-549

Dear CDH, Cloudera Manager, and Cloudera Impala users,   We are pleased to announce the release of Cloudera Enterprise 5.0.1 (CDH 5.0.1 and Cloudera Manager 5.0.1) and Cloudera Manager 4.8.3.   Cloudera Enterprise 5.0.1 This release is focused on fixing key bugs and includes the following:   Component updates Impala 1.3.1 Cloudera Manager HDFS NFS gateway does not work on all Cloudera-supported platforms Replace YARN_HOME with HADOOP_YARN_HOME during upgrade Insufficient password hashing in Cloudera Manager Upgrade to Cloudera Manager 5.0.0 from SLES older than Service Pack 3 with PostgreSQL older than 8.4 fails MR1 to MR2 import fails on a secure cluster After upgrade from CDH 4 to CDH 5, Oozie is missing workflow extension schemas CDH Fixes SOLR-5608 - Frequently reproducible failures in CollectionsAPIDistributedZkTest#testDistribSearch HIVE-6648 - Fixed permission inheritance for multi-partitioned tables HIVE-6740 - Fixed addition of Av

Dear CDH, Cloudera Manager, and Cloudera Impala users, We are pleased to announce the general availability of Cloudera Impala 1.3.1 for use with CDH 4.   Impala 1.3.1 contains stability enhancements and the following new features: Improved memory usage under contention including higher concurrency, high numbers of Parquet columns, or low memory limits Additional SQL built-in functions Note that Impala 1.3.1 does not work with Cloudera Manager 4.7 or earlier. If your environment is managed by Cloudera Manager 4.7 or before, you should upgrade to Impala 1.3.1 and Cloudera Manager 4.8 together. For more details on new Impala features, please see “New Features in Impala”: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Cloudera-Impala-Release-Notes/cirn_new_features.html “Installing and Using Cloudera Impala” guide: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/Installing-and-Using-Impala.

We are thrilled to announce the GA release of Cloudera Enterprise 5 (comprising CDH 5.0 and Cloudera Manager 5.0).    When it became generally available 18 months ago, Cloudera Enterprise 4 was widely recognized to be more flexible, more scalable, and less expensive than traditional data management platforms; with Cloudera Enterprise 5, we can also say it is faster (HDFS caching FTW!), more functional (includes new components such as Apache Spark and Parquet), and better integrated (for example, Cloudera Impala and Cloudera Search are now both inside CDH). Furthermore, in an impressive demonstration of ecosystem growth, 100 Cloudera partners invested technical resources to certify on Cloudera Enterprise 5 before its release.   Cloudera Enterprise contains many new components and features, with the major ones listed below. (See CDH 5 Release Notes and Cloudera Manager 5 Release Notes for full details.) Thanks to everyone who partici

Dear Cloudera users,  We are pleased to announce the general availability of the Cloudera Connector Powered by Teradata and the Cloudera Connector for Netezza version 1.2. This release has many updates including: For Cloudera Connector Powered by Teradata: Upgraded TDCH to version 1.2.1 Added support for Avro file format Added support for Incremental import Added support for --where argument Added support for Hive import Added support for Importing all tables using import-all-tables Added support for Query Bands Added new import method split.by.amp (supported only on Teradata 14.10 and higher) For Cloudera Connector for Netezza: Added --nz-uploaddir for uploading logs to HDFS directory Added --nz-ctrlchars to enable CTRLCHARS option for the Netezza external table Both connectors are released in editions compatible with CDH 4 and CDH 5. See download page for further details.   Please note that the Cloudera Connector for Teradata is not supported on CDH 5.  Y

We're happy to announce the release of Kite 0.12.0.This release includes a new MapReduce InputFormat and OutputFormat for reading from and writing to Kite Datasets. There are also several Morphlines updates including a new set of examples, and bug fixes and minor improvements.For more details, see the release notes, and the documentation.

Dear Cloudera Impala and CDH users,     We are pleased to announce the general availability of Cloudera Impala 1.2.4. This maintenance release fixes issues with the Catalog Service as well as a few other bugs, and enables metadata refreshes without full server permissions.   Note that Impala 1.2.4 does not work with Cloudera Manager 4.7 or earlier. If your environment is managed by Cloudera Manager, you should upgrade to Impala 1.2.4 and Cloudera Manager 4.8 together.   For more details on new Impala features, please see “New Features in Impala”.   “Installing and Using Cloudera Impala” guide: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/Installing-and-Using-Impala.html   As always, we welcome your feedback. Please send your comments and suggestions through our new community forums.  You can also file bugs in the Impala project at issues.cloudera.org.  

We are pleased to announce the general availability of CDH 4.6, Cloudera Manager 4.8.2 and Cloudera Search 1.2.  Details on each release are below.   CDH 4.6 This is an update to the CDH 4 platform and provides a considerable amount of bug-fixes and stability enhancements. Along with these fixes, we have also included some new features, most notable of which are: Rebase to HBase .94.15 HUE-1664 - URL Redirection Whitelist For more details on new features, see “New Features in CDH 4”.   Refer to the “CDH 4 Quick Start Guide” or the “CDH 4 Installation Guide” for more information on how to install this update release.   As always, we are happy to hear your feedback. Please send your comments and suggestions through our new community forums. You can also file bugs through the Distro project on issues.cloudera.org.     Cloudera Manager 4.8.2 Cloudera Manager 4.8.2 is a maintenance release and has a number of bug-fixes and usability improvements.&nbs

Cloudera has released the Beta 2 version of Cloudera Enterprise 5 (comprises CDH 5.0.0 and Cloudera Manager 5.0.0).  This release (download) contains a number of new features and component versions including the ones below: New Components Apache Spark 0.9 Apache Crunch 0.9 Parquet 1.2.5 Kite SDK  0.10 Apace Avro 1.7.5 Apache Hadoop Rebase to Hadoop 2.2.0+ HDFS HDFS Caching NFS gateway Cloudera Manager Support for Apache Spark Oozie and YARN Resource Manager High Availability Extensibility to support add-on services Enhancements to Monitoring & Charts Apache HBase Rebase to HBase 0.96.1.1 Impala Rebase to Impala 1.2.3 Apache Flume New Kite Dataset Sink Apache Hive Hive 0.12 Rebase Improved JDBC spec coverage SSL encryption support on non-kerberos authentication Native Parquet support Apache Sqoop Rebase to Sqoop 1.99.3 Apache Oozie Support for CRON like scheduling Hue Single Sign On (SSO) support Graphical facets in Search application Result graphing for

We are pleased to announce the release of Kite 0.11.0.The main new feature in this release is the views API for working with a subset of a dataset using logical constraints such as field matching or ranges. There are also assorted Morphlines updates, and several bug fixes and minor improvements.For more information, see the release notes, and the documentation.

Dear CDH and Cloudera Manager users,     We are pleased to announce the general availability of Cloudera Manager 4.8.1.   Cloudera Manager 4.8.1 is a maintenance release and has a number of bug-fixes and usability improvements.   For more details, see the release notes: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Cloudera Manager4Ent/latest/Cloudera-Manager-Release-Notes/Cloudera-Manager-Release-Notes.html   All of the Cloudera Manager documentation can be found here: http://www.cloudera.com/content/support/en/documentation/manager/cloudera-manager-v4-latest.html   You can download this latest Cloudera Manager here or you can upgrade from an existing installation of Cloudera Manager.   Installation and upgrade documentation can be found here.

Cloudera is pleased to announce the immediate availability of its first release of Apache Spark for Cloudera Enterprise (comprising CDH and Cloudera Manager).   Spark was created and contributed to the Apache Software Foundation by UC Berkeley, and it has quickly gained adoption for machine learning, interactive analytics, and streaming analytics over large datasets. It features a general programming model for writing applications by composing arbitrary operators, such as mappers, reducers, joins, group-bys, and filters. Spark keeps track of the data that each of the operators produces, enabling applications to reliably store this data in memory, which makes it ideal for low-latency computations and efficient iterative algorithms. Spark applications can be up to 100x faster and require writing 2x to 10x less code than equivalent MapReduce applications.    Cloudera provides enterprise support for Spark through Cloudera Enterprise Flex Edition (as an optiona