Dear Cloudera Manager users,   We are pleased to announce the release of Cloudera Manager 5.1.1 and 4.8.4.  These are maintenance releases and have a number of bug fixes.   Cloudera Manager 4.8.4 Bug Fixes:   Directory quota in Cloudera Manager shows incorrect values. Activity Monitor in Cloudera Manager doesn't request new http service ticket. Many Hive replications running at the same time can cause some of them to fail with "Read timed out" errors. Multiple Hive replications running at the same time can cause some of them to fail with "command succeeded, but unable to fetch results" errors. This has been noted to occur with as few as 20 replications. Multiple Hive replications started at exactly the same time (typically triggered by replication schedules) can cause one or more of them to complete successfully without actually replicating the desired metadata. Replications can be affected by other replications or commands running at the same time.  For more details, see the Cloudera Manager 4.8.4 release notes: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Cloudera Manager4Ent/latest/Cloudera-Manager-Release-Notes/Cloudera-Manager-Release-Notes.html   All of the Cloudera Manager 4.8.4 documentation can be found here: http://www.cloudera.com/content/support/en/documentation/manager/cloudera-manager-v4-latest.html   Cloudera Manager 5.1.1 Bug Fix:   Add/Upgrade Hosts wizards install incorrect JCE policy files if Java 7 is already installed on RPM based distros.  For more details, see the Cloudera Manager 5.1.1 release notes: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Release-Notes/Cloudera-Manager-Release-Notes.html   All of the Cloudera Manager 5.1.1 documentation can be found here: http://www.cloudera.com/content/support/en/documentation/manager/cloudera-manager-v5-latest.html   You can download this latest Cloudera Manager here or you can upgrade from an existing installation of Cloudera Manager. Installation and upgrade documentation can be found here for Cloudera Manager 4.8.4 and here for Cloudera Manager 5.1.1.   As always, we are happy to hear your feedback. Please send your comments and suggestions to scm-user@cloudera.org or through our community forums. ... View more

Cloudera is pleased to announce support for Accumulo 1.6.0 on both CDH 4 and CDH 5.   As of version 5.1.0, Cloudera Manager now ships with support to deploy and manage Accumulo 1.6.0 on CDH 4.6.0 or later and CDH 5.1.0 or later.   * Docs: http://tiny.cloudera.com/accumulo-docs * CM 5.1.0 Download: http://tiny.cloudera.com/cm-5.1.0 * Accumulo 1.6.0 on CDH 5 (CDH5.1.0 or later): http://archive.cloudera.com/accumulo-c5/ * Accumulo 1.6.0 on CDH 4 (CDH4.6.0 or later): http://archive.cloudera.com/accumulo/ ... View more

Dear CDH, Cloudera Manager, Impala and Search Users,     We are thrilled to announce the GA release of Cloudera Enterprise 5.1 (CDH 5.1 and Cloudera Manager 5.1),  Cloudera Navigator 2.0, Impala 1.4 for CDH 4, Impala ODBC v2.5.17 and Hive ODBC v2.5.10.   Cloudera Enterprise 5.1 Cloudera Enterprise 5.1 contains a number of new features and component versions including the ones below: Apache Hadoop HDFS Extended ACL Support NFS Enhancements HTTPS support YARN YARN + Impala GA: CDH 5.1 now supports three modes of Resource Management: Static Partitioning (CDH 4.3+) Admission Control (CDH 5.0+) Dynamic Prioritization (CDH 5.1+) As part of Dynamic Prioritization, Llama now supports HA Configuration similar to other HA elements (Active-Passive) Apache HBase HBase 0.98.1 rebase Improved WAL write performance Reverse scans MapReduce over snapshots Apache Spark Rebase to Spark 1.0 Spark Streaming integration with Kerberos Application History Server improves monitoring capabilities Sparse Vector Support in MLLib Improvements to Avro integration Simplified job submission to YARN cluster Security: Authentication of all Spark communications Apache Sentry Rebase to 1.3 Support for Grant/Revoke statements via beeline (HiveServer2) Policy files no longer required Sentry fixes: job tracking in Hue, HDFS permissions inheritance Impala Release of  Impala 1.4.0 YARN-integrated Resource Management is GA DECIMAL across Hive, Impala, Avro, Parquet, and text file formats Integration with HDFS Caching Fixed folder inheritance issues for better framework interoperability (IMPALA-827) LDAPS Support for more secure AD username/password authentication ORDER BY without LIMIT Performance improvements for COMPUTE STATS and queries DDL support for creating a new table from an existing Parquet file schema DDL support for creating Avro tables Additional built-in functions like TRUNC, EXTRACT, and stats functions Apache Hive DECIMAL across Hive, Impala, Avro, Parquet, and text file format Fixed folder inheritance issues for better framework interoperability (HIVE-6892) Cloudera Search Document Level Security Parquet Support Search QuickStart Document and Script HBase Lily Indexer update Apache Oozie Added ability to submit Sqoop jobs from Oozie CLI (Knox competitive) Completed testing of JMS/Email notification for workflow and Coordinator action status changes Completed testing of SLA notifications Replaced/Improved Metrics instrumentation (for Cloudera Manager monitoring) LAST_ONLY execution mode now works correctly Cloudera Manager Fine grain access for Cloudera Manager Two additional roles in Cloudera Manager - Operator and Configurator Impala Llama HA support Security Updates Integrate directly with Active Directory to set up kerberos principals needed for CDH daemons New wizard to add kerberos to an existing unsecure cluster Manage and deploy kerberos client configurations (krb5.conf). Sentry support for Grant/Revoke Support for SSL Manage Hadoop SSL related configurations Monitor Hadoop services when SSL is enabled Monitoring Updates to Oozie monitoring New Hive Metastore Canary Hue Rebase to 3.6 Search App v2 100% dynamic dashboards Drag and drop dashboard builder Text, Timeline, Pie, Line, Bar, Map, Filters, Grid, and HTML Widgets Solr Index Designer (basic up-and-running wizard, from a file in HDFS) View support for Snappy, Avro, Parquet files Multi NT domain LDAP Nested Groups Impala HA Close commands for Impala/Hive sessions and queries Other Rebases Apache Mahout - 0.9 Apache Crunch - 0.10 Flume 1.5.0   We look forward to you trying it out using the information below:   Download Cloudera Enterprise from: cloudera.com/downloads View the documentation: CDH 5 Documentation Cloudera Manager 5 Documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external jira projects on issues.cloudera.org.     Cloudera Navigator 2.0 Cloudera Navigator is a fully-integrated governance, compliance, and risk management solution for Hadoop. Cloudera Navigator provides comprehensive metadata, lineage, and auditing support across the enterprise data hub.   Unified technical and business metadata Consolidate technical metadata for Hadoop files and tables, plus add custom tags and comments so that you can easily track, annotate and classify data in alignment with business rules. Lineage View upstream and downstream column-level lineage in an easy-to-follow graph so that you can quickly identify the origin of a data set and its impact on downstream analysis. Supports Hive, Pig, Sqoop 1, MapReduce, YARN, Oozie Auditing View and summarize all data access attempts with a simple, queryable interface so you can quickly identify outliers and security breaches Supports Hive, Impala, HBase, HDFS, Sentry Security Cloudera Navigator includes Navigator Encrypt and Navigator Key Trustee, formerly known as Gazzang zNcrypt and Gazzang zTrustee. These products provide enterprise-grade encryption and key management. Links Product Information http://www.cloudera.com/content/cloudera/en/products-and-services/cloudera-enterprise/cloudera-navigator.html Documentation: http://www.cloudera.com/content/support/en/documentation/cloudera-navigator/cloudera-navigator-v2-latest.html   Impala 1.4 on CDH4 Impala 1.4 for CDH4 contains features that are included in Cloudera Enterprise 5.1 which are supported by CDH4: Fixed folder inheritance for Sentry (IMPALA-827) LDAPS Support for more secure AD username/password authentication ORDER BY without LIMIT DDL support for creating a new table from an existing Parquet file schema DDL support for creating Avro tables Fixed folder inheritance issues for better framework interoperability (IMPALA-827) Performance improvements for COMPUTE STATS and queries Additional built-in functions like TRUNC, EXTRACT, and stats functions  Note that Impala 1.4 does not work with Cloudera Manager 4.7 or earlier. If your environment is managed by Cloudera Manager 4.7 or earlier, you should upgrade to Impala 1.4 and Cloudera Manager 4.8 together.   For more details on new Impala features, please see “New Features in Impala” for CDH 4: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Cloudera-Impala-Release-Notes/cirn_new_features.html   “Installing and Using Cloudera Impala” guide for CDH 4: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/Installing-and-Using-Impala.html   As always, we are happy to hear your feedback. Please send your comments and suggestions to impal...@cloudera.org or through our new community forums.  You can also file bugs in the Impala project at issues.cloudera.org     Impala ODBC 2.5.17 and Hive ODBC 2.5.10 The new ODBC drivers include the following new functionality: Impala ODBC fixes the default fetch size for faster client returns Improved Mac OSX client installation We look forward to you trying it out using the links below: Impala ODBC v2.5.17 Hive ODBC v2.5.10 ... View more

Version 0.15.0 contains the following notable changes:   Kite artifacts are built against Apache Hadoop 2 and related projects, and are now available in Maven Central. Added new introduction and concepts documentation. Added a new Datasets convenience class for opening and working with Datasets, superseding DatasetRepositories. Deprecated partition related methods in Dataset in favor of the views API. Added a CLI copy task for copying datasets and also for dataset format conversion and data compaction. Added an application parent POM that makes it easy to use Kite in a Maven project. The examples now use this parent POM. Updated to Crunch 0.10.0 Morphlines Library Added morphline command that parses an InputStream that contains protobuf data: readProtobuf (Rober Fiser via whoschek) Added morphline command that extracts specific values from a protobuf object, akin to a simple form of XPath: extractProtobufPaths (Rober Fiser via whoschek) Added morphline command that removes all record fields for which the field name matches a blacklist but not a whitelist: removeFields Added optional parameters  maxCharactersPerRecord  and  onMaxCharactersPerRecord  to morphline command readCSV Upgraded kite-morphlines-maxmind module from maxmind-db-0.3.1 to bug fix release maxmind-db-0.3.3 Upgraded kite-morphlines-core module from metrics-0.3.1 to bug fix release metrics-0.3.2 The full change log is available from JIRA.     ... View more

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.0.3 (CDH 5.0.3 and Cloudera Manager 5.0.2).   This release is focused on fixing key bugs and includes the following: CDH Fixes FLUME-2245  - HDFS files with errors unable to close FLUME-2416 - Use CodecPool in compressed stream to prevent leak of direct buffers HBASE-10871  Indefinite OPEN/CLOSE wait on busy RegionServers HDFS-5891 - WebHDFS should not try connecting the DataNode during redirection HDFS-6021 - NPE in FSImageFormatProtobuf upgrading from layout -52 to -53 HDFS-6077 - running slive with WebHDFS on secure HA cluster fails with unknown host exception HDFS-6340 - DataNode can't finalize upgrade HDFS-6475 - WebHDFS clients fail without retry because incorrect handling of StandbyException HDFS-6510 - WebHDFS clients clear the delegation token on retry (for HA), thus failing retry requests HDFS-6527 - Edit log corruption due to deferred INode removal HDFS-6563 - NameNode cannot save fsimage in certain circumstances when snapshots are in use HUE-1928  - [beeswax] HiveServer2 supports pass-through LDAP authentication HUE-2085 - [core] Update migration dependencies HUE-2184 - [core] Connect to Oracle via Service Name HUE-2192 - [core] Create parameter to choose LDAP username for HiveServer2 HUE-2193 - [beeswax] HiveServer2 pass-through LDAP authentication at thrift level OOZIE-1621 - Add proper error code and error message for sharelib exceptions. OOZIE-1890 - Make oozie-site empty and reconcile defaults between oozie-default and the code OOZIE-1907 - DB upgrade from 3.3.0 to trunk fails on derby SOLR-5593 - shard leader loss due to ZK session expiry SOLR-5915 - Cannot set parserImpl=... with PreAnalyzedField SOLR-6161 - OutOfMemoryError Not Thrown in sendError YARN-1550 - NPE in FairSchedulerAppsBlock#render YARN-2155 - FairScheduler: Incorrect threshold check for preemption We look forward to you trying it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: CDH 5 Release Notes CDH 5 Documentation Cloudera Manager Release Notes Cloudera Manager 5 Documentation As always, we are happy to hear your feedback in these forums. You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.0.2 (CDH 5.0.2 and Cloudera Manager 5.0.2). This release is focused on fixing key bugs and includes the following:   Cloudera Manager Cloudera Manager Impala Query Monitoring works with Impala 1.3.1 CDH Fixes HADOOP-10556: Add toLowerCase support to auth_to_local rules for service name HADOOP-10638: Updating hadoop-daemon.sh to work as expected when nfs is started as a privileged user. HADOOP-10639: FileBasedKeyStoresFactory initialization is not using default for SSL_REQUIRE_CLIENT_CERT_KEY HADOOP-10658: SSLFactory expects truststores being configured HBASE-6990: Pretty print TTL HBASE-10312: Flooding the cluster with administrative actions leads to collapse HBASE-10371: Compaction creates empty hfile, then selects this file for compaction and creates empty hfile and over again HDFS-6326: WebHdfs ACL compatibility is broken HIVE-6913: Hive unable to find the hashtable file during complex multi-staged map join HIVE-5380: Non-default OI constructors should be supported for backwards compatibility PIG-3677: ConfigurationUtil.getLocalFSProperties can return an inconsistent property set YARN-2073: Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free We look forward to you trying it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: CDH 5 Release Notes CDH 5 Documentation Cloudera Manager Release Notes Cloudera Manager 5 Documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org.   Cloudera Team ... View more