Dear CDH, Cloudera Manager, Impala and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.2.1 (CDH 5.2.1, Cloudera Manager 5.2.1, and Cloudera Navigator 2.1.1)   This release is focused on fixing key bugs and includes the following:   CDH Fixes Oozie: Using cron-like syntax for Coordinator frequencies could result in duplicate actions in certain cases; this is now fixed (OOZIE-2063) YARN: Handle app-recovery failures gracefully (YARN-2010) Impala: Memory leak with string functions (IMPALA-1397) Impala: IllegalStateException when inserting results of a window function (IMPALA-1400) Impala: Read errors with Parquet files (IMPALA-1401) Impala: Regex functions don’t accept shorthand such as \d (IMPALA-1410) Impala: Queries fail with metastore exception after upgrade and compute stats (IMPALA-1416) Impala: Crashes due to bug in ClientCacheHelper (IMPALA-1445) Cloudera Manager Fixed metric collection for CDH 5.0 HDFS daemons. Fixed OutOfMemory crashes on Thrift servers in Reports Manager and Event Server. Replication commands respects JAVA_HOME if an override has been provided for it. Fixed ZooKeeper connection leaks from HBase clients used by the Service Monitor. For parcel-based installations, user home directories are created with umask 022 (instead of the user add default of 077) A new health check has been added to indicate if HDFS rolling upgrade has not been finalized. Cloudera Navigator LDAP lookups in Active Directory to resolve group membership are now working. Dropping a hive table and creating a view with same name or vice versa no longer raises an error. HDFS extraction now works after upgrading CDH from 5.1 to 5.2 Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata We look forward to you trying it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/ content/support/en/downloads. html View the documentation: CDH 5 Release Notes Cloudera Manager Release Notes Cloudera Navigator Release Notes Cloudera Documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

We're pleased to announce the release of Cloudera Enterprise 5.2 (comprising CDH 5.2, Cloudera Manager 5.2, Cloudera Director 1.0, and Cloudera Navigator 2.1).   This release reflects our continuing investments in Cloudera Enterprise's main focus areas, including security, integration with the partner ecosystem, and support for the latest innovations in the open source platform (including Impala 2.0, its most significant release yet, and Apache Hive 0.13.1). It also includes a new product, Cloudera Director, that streamlines deployment and management of enterprise-grade Hadoop clusters in cloud environments; new component releases for building real-time applications; and new support for significant partner technologies like EMC Isilon. Furthermore, this release ships the first results of joint engineering with Intel, including WITH GRANT OPTION for Hive and Impala and performance optimizations for MapReduce.   Here are some of the highlights (incomplete; see the respective Release Notes for CDH, Cloudera Manager, and Cloudera Navigator for full lists of features and fixes):   Security Via Apache Sentry (incubating) 1.4, GRANT and REVOKE statements in Impala and Hive can now include WITH GRANT OPTION, for delegation of granting and revoking privileges (joint work with Intel under Project Rhino). Hue has a new Sentry UI that supports policy management for visually creating/editing roles in Sentry and permissions on Files in HDFS . Kerberos authentication is now supported in Apache Accumulo. Impala, authentication can now be done through a combination of Kerberos and LDAP. Data Management and Governance Cloudera Navigator 2.1 features a brand new auditing UI that is unified with lineage and discovery, so you now have access to all Navigator functionality from a single interface. Navigator 2.1 includes role-based access control so you can restrict access to auditing, metadata and policy management capabilities We’re also shipping a beta policy engine in Navigator 2.1. Targeted to GA by year-end, the policy engine allows you to set up rules and notifications so you can classify data as it arrives and integrate with data preparation and profiling tools. Try it out and let us know what you think! And we’ve added lots of top-requested enhancements, such as Sentry auditing for Impala and integration with Hue. Cloud Deployment Cloudera Director is a simple and reliable way to deploy, scale, and manage Hadoop in the cloud (initially for AWS) in an enterprise-grade fashion. It’s free to download and use, and supported by default for Cloudera Enterprise customers. Features include: Simple UI for self-service cluster spin up/teardown Dynamic scaling for spiky workloads Simple cloning of clusters Cloud blueprints for repeatable deployments Third-party software deployment within same workflow Support for custom, workload-specific deployments Support for complex cluster topologies Minimum size cluster when capacity constrained Multi-cluster dashboard Instance tracking for account billing Real-Time Architecture   Rebase on Apache HBase 0.98.6 Cell-level ACLs for fine-grained access control of data in HBase now supported Backported improvements to get and put request scheduling and throttling that provide basic QoS for multi-tenant HBase tables and clusters. Lets some production and real-time workloads take priority over ad hoc and analytic jobs. Backported patches that make Offheap Block Cache (aka bucket cache) production-ready. Now you can use large amounts of memory for read caching without the GC penalties of the past. Bucket cache is now the default. Backported authentication of clients accessing HBase via the HBase Thrift Proxy. Rebase on Apache Spark/Streaming 1.1 Rebase on Impala 2.0 Cloudera Search now provides Spark-indexing - iterative, fast index design distributed pivot facets ability to expire documents node fail recovery support for deep paging and for multithreaded faceting Apache Sqoop now supports import into Apache Parquet (incubating) file format Apache Kafka integration with CDH is now incubating in Cloudera Labs; a Kafka-Cloudera Labs parcel (unsupported) is available for installation. Integration with Flume via special Source and Sink have been provided. Impala 2.0   Disk-based query processing: enables large queries to "spill to disk" if their in-memory structures are larger than the currently available memory. (Note that this feature only uses disk for the portion that doesn't fit in the available memory.) Greater SQL compatibility: SQL 2003 analytic (window) functions, support for legacy data types (such as CHAR and VARCHAR), better compliance with SQL standards (WHERE, EXISTS, IN), and additional vendor-specific SQL extensions. Impala 2.0 is now also available for CDH 4. New Open Source Releases and Certifications   Cloudera Enterprise 5.2 includes multiple new component releases:   Apache Avro 1.7.6 Apache Crunch 0.11 Apache Hadoop 2.5 Apache HBase 0.98.6 Apache Hive 0.13.1 Apache Parquet (incubating) 1.5 / Parquet-format 2.1.0 Apache Sentry (incubating) 1.4 Apache Spark 1.1 Apache Sqoop 1.4.5 Impala 2.0 Kite SDK 0.15.0 ...with new certifications on:   Filesystems: EMC Isilon OSs: Ubuntu 14.04 (Trusty) Java: Oracle JDK1.7.0_67   Over the next few weeks, we’ll publish blog posts that cover some of these and other new features in detail. In the meantime: Download Cloudera Enterprise 5.2 Explore documentation As always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs via issues.cloudera.org. ... View more

We're pleased to announce the release of Kite SDK 0.17.0. This release updates the examples to CDH 5, defaults Parquet to the non-durable mode from 0.14 and prior, adds support for namespaces, and adds a kite-minicluster for easier development and integration testing against single-node Hadoop deployments. For more details see the release notes and the documentation. ... View more

Hello CDH and Impala Users,   We are pleased to announce the release of version 2.5.12 of the ODBC Driver for Apache Hive and version 2.5.20 of the ODBC driver for Impala.  These versions contain bug fixes including one that affected the decimal data type. These drivers work for previous versions of HiveServer2 in CDH 4.1 or higher and Impala 1.0 or higher.   Getting started with the Cloudera ODBC Drivers:   Read the Cloudera ODBC 2.5 Driver for Impala release notes and installation guide Read the Cloudera ODBC 2.5 Driver for Apache Hive release notes and  installation guide Download the connector from the Cloudera Connectors page As always, we are happy to hear your feedback. Please send your comments and suggestions to cdh-user@cloudera.org or post to our new Community Forums.   Kind regards, The Cloudera Team ... View more

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.1.3.   Cloudera Enterprise 5.1.3 This release is focused on fixing key bugs and includes the following. CDH Fixes HADOOP-11035 - distcp on mr1(branch-1) fails with NPE using a short relative source path. HBASE-11349 - [Thrift] support authentication/impersonation HBASE-11446 - Reduce the frequency of RNG calls in SecureWALCellCodec#EncryptedKvEncoder HBASE-11457 - Increment HFile block encoding IVs accounting for ciper's internal use HBASE-11474 - [Thrift2] support authentication/impersonation HBASE-11565 - Stale connection could stay for a while HBASE-11627 - RegionSplitter's rollingSplit terminated with "/ by zero", and the _balancedSplit file was not deleted properly HBASE-11788 - hbase is not deleting the cell when a Put with a KeyValue, KeyValue.Type.Delete is submitted HBASE-11828 - callers of SeverName.valueOf should use equals and not == HDFS-4257 - The ReplaceDatanodeOnFailure policies could have a forgiving option HDFS-6776 - Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work HDFS-6908 - incorrect snapshot directory diff generated by snapshot deletion HUE-2247 - [Impala] Support pass-through LDAP authentication HUE-2273 - [desktop] Blacklisting apps with existing document will break home page HUE-2295 - [librdbms] External oracle DB connection is broken due to a typo HUE-2318 - [desktop] Documents shared with write group permissions are not editable HIVE-5087 - Rename npath UDF to matchpath HIVE-6820 - HiveServer(2) ignores HIVE_OPTS HIVE-7635 - Query having same aggregate functions but different case throws IndexOutOfBoundsException IMPALA-958 - Excessively long query plan serialization time in FE when querying huge tables IMPALA-1091 - Improve TScanRangeLocation struct and associated code OOZIE-1989 - NPE during a rerun with forks YARN-1458 - FairScheduler: Zero weight can lead to livelock Cloudera Manager Adding and upgrading hosts allows users to skip installing default JDK that ships with Cloudera Manager. Improved speed and heap usage when deleting hosts on cluster with long history. When there are multiple clusters, each cluster's topology files and validation for legal topology is limited to hosts in that cluster. Most commands will now fail up front if the cluster's topology is invalid. For users using Oracle databases, the size of the statement cache has been reduced, to help with memory consumption. Improvements to memory usage of "cluster diagnostics collection" for large clusters. Cloudera Navigator HBase auditing initialization failure can prevent region opening indefinitely. We look forward to you trying it out using the information below:   Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: CDH 5 Release Notes CDH 5 Documentation Cloudera Manager Release Notes Cloudera Manager 5 Documentation Cloudera Navigator Documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

Dear CDH and Cloudera Manager users,   We are pleased to announce the release of Cloudera Enterprise 5.1.2 (CDH 5.1.2, Cloudera Manager 5.1.2, Cloudera Navigator 2.0.1) and CDH 5.0.4.   Cloudera Enterprise 5.1.2 This release is focused on fixing key bugs and includes the following.   CDH Fixes FLUME-2438 - Make Syslog source message body configurable HBASE-11052 - Sending random data crashes thrift service HBASE-11143 - Improve replication metrics HBASE-11609 - LoadIncrementalHFiles fails if the namespace is specified HDFS-6114 - Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr HDFS-6640 - [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).). HDFS-6703 - NFS: Files can be deleted from a read-only mount HDFS-6788 - Improve synchronization in BPOfferService with read write lock HDFS-6825 - Edit log corruption due to delayed block removal HUE-2211 - [search] Twitter and Jobs example do not load properly HUE-2223 - [beeswax] Bigints are rounded on result tab HUE-2232 - [search] Examples don't install with MySql HIVE-5515 - Writing to an HBase table throws IllegalArgumentException, failing job submission HIVE-6495 - TableDesc.getDeserializer() should use correct classloader when calling Class.forName() HIVE-7450 - Database should inherit perms of warehouse dir IMPALA-1093 - Impalad catalog updates can fail with error: "IllegalArgumentException: fromKey out of range" at com.cloudera.impala.catalog.CatalogDeltaLog IMPALA-1107 - Update HS2 client API. IMPALA-1131 - "Total" time counter does not capture all the network transmit time IMPALA-1142 - Support specifying a custom AuthorizationProvider in Impala IMPALA-1149 - Impala will crash when reading certain Avro files containing bytes data MAPREDUCE-5966 - MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons MAPREDUCE-5979 - FairScheduler: zero weight can cause sort failures MAPREDUCE-6012 - DBInputSplit creates invalid ranges on Oracle OOZIE-1920 - Capture Output for SSH Action doesn't work SENTRY-363 - CTAS from view is requiring select on underlying table YARN-2273 - NPE in ContinuousScheduling thread when we lose a node YARN-2274 - FairScheduler: Add debug information about cluster capacity, availability and reservations YARN-2313 - Livelock can occur in FairScheduler when there are lots of running apps YARN-2352 - FairScheduler: Collect metrics on duration of critical methods that affect performance YARN-2359 - Application hangs when it fails to launch AM container Cloudera Manager New SAML configuration option to specify the binding protocol to be used for AuthNResponses sent from the IDP to Cloudera Manager. Host version detection logic fixed in Upgrade wizard when upgrading from package or a de-activated CDH4 parcel to CDH 5 parcels. AWS Installation wizard is fixed to work with Java 7u55 BDR Replications can run in parallel with other replications. Cloudera Navigator Masking of personally identifiable information (PII) in query strings that appear in audit events and lineage. REST API support for registering business metadata for entities before they appear in Navigator.   CDH 5.0.4 This release is focused on fixing key bugs and includes the following.   FLUME-2438 - Make Syslog source message body configurable HBASE-11609 - LoadIncrementalHFiles fails if the namespace is specified HDFS-6044 - Add property for setting the NFS look up time for users HDFS-6529 - Trace logging for RemoteBlockReader2 to identify remote datanode and file being read HDFS-6618 - FSNamesystem#delete drops the FSN lock between removing INodes from the tree and deleting them from the inode map HDFS-6622 - Rename and AddBlock may race and produce invalid edits HDFS-6640 - [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).). HDFS-6647 - Edit log corruption when pipeline recovery occurs for deleted file present in snapshot HDFS-6703 - NFS: Files can be deleted from a read-only mount HDFS-6788 - Improve synchronization in BPOfferService with read write lock HIVE-5515 - Writing to an HBase table throws IllegalArgumentException, failing job submission HIVE-7459 - Fix NPE when an empty file is included in a Hive query that uses CombineHiveInputFormat HUE-2166 - [core] Oracle database support in doc model HUE-2249 - [jobsub] DB migration problems from 2 to 3.6 IMPALA-1019 - Failed DCHECK in disk-io-mgr-reader-context.cc:174] num_used_buffers_ < 0: #used=-1 during cancellation HDFS cached data MAPREDUCE-5966 - MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons MAPREDUCE-5979 - FairScheduler: zero weight can cause sort failures OOZIE-1920 - Capture Output for SSH Action doesn't work SPARK-1930 - The Container is running beyond physical memory limits, so as to be killed. YARN-2061 - Revisit logging levels in ZKRMStateStore YARN-2132 - ZKRMStateStore.ZKAction#runWithRetries doesn't log the exception it encounters Note: There is no CDH 5.1.1 release. This skip in the CDH 5.x sequence allows the CDH and CM components of Cloudera Enterprise 5.1.2 to have consistent numbering.   We look forward to you trying it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html View the documentation: CDH 5 Release Notes CDH 5 Documentation Cloudera Manager Release Notes Cloudera Manager 5 Documentation Cloudera Navigator Documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

We're pleased to announce the release of Kite SDK 0.16.0. This release adds support for Apache Spark, adds a CLI transform command for dataset-to-dataset ETL, and adds a CDH 5 parent POM for building Kite applications targeting CDH 5.   For more details see the release notes and the documentation. ... View more