Dear Cloudera users,    We are pleased to announce the general availability of the Cloudera Connector Powered by Teradata 1.3. This release has many updates including:   New features such as: Upgraded Teradata Connector for Hadoop to version 1.3.3 Parcel distribution now contains Teradata JDBC driver and one don't have to download one manually Added support for query import into Avro file format Other notable changes: Import method multiple.fastload has been removed. Connector is released in editions compatible with CDH 4 and CDH 5. See download page for further details. For more details on new features and usage of Cloudera Connector Powered by Teradata, see: Release Notes Cloudera Connector Powered by Teradata version 1.3 Cloudera Connector Powered by Teradata User Guide, version 1.3 As always, we welcome your feedback. Please send your comments and suggestions through our new community forums.  You can also file bugs in the CDH project at issues.cloudera.org ... View more

Hi Cloudera users,   We're pleased to announce the release of Cloudera Enterprise 5.3 (comprising CDH 5.3, Cloudera Manager 5.3, and Cloudera Navigator 2.2).   This release continues the drumbeat for security functionality in particular, with HDFS encryption (jointly developed with Intel under Project Rhino) now recommended for production use. This feature alone should justify upgrades for security-minded users (and an improved CDH upgrade wizard makes that process easier).   Here are some of the highlights (incomplete; see the respective Release Notes for CDH, Cloudera Manager, and Cloudera Navigator for full lists of features and fixes): Security Folder-level HDFS encryption (in addition to storage, management, and access to encryption zone keys) is now a production-ready feature (HDFS-6134). This feature integrates with Navigator Key Trustee so that encryption keys can be securely stored separately from the data, with all the enterprise access and audit controls required to pass most security compliance audits such as PCI. The Cloudera Manager Agent can now be run as a single configured user when running as root is not permitted. In Apache Sentry (incubating), data can now be shared across Impala, Apache Hive, Search, and other access methods such as MapReduce using only Sentry permissions. A Sentry bug that affected CDH 5.2 upgrades has been patched (SENTRY-500). Data Management and Governance In Cloudera Navigator 2.2, policies are now generally available and enabled by default. Policies let you set, monitor and enforce data curation rules, retention guidelines, and access permissions. They also let you notify partner products, such as profiling and data preparation tools, whenever there are relevant changes to metadata. Navigator 2.2’s REST API now supports user-defined relations. Using these new APIs, you can augment Navigator’s automatically-generated lineage with your own column-level lineage. This is particularly useful for custom MapReduce jobs that run on structured data sources. Navigator 2.2 also features many top-requested enhancements, including metadata search auto-suggest and a number of other usability improvements. Cloud Deployments Cloudera Enterprise 5.3 is now a first-class citizen with respect to deployments on Microsoft Azure. Apache Hadoop gets a new S3-native filesystem for improved performance on AWS (HADOOP-10400). Real-Time Architecture Apache Flume now includes an Apache Kafka Channel for tighter integration (FLUME-2500) . Apache HBase performance is significantly improved thanks to updated defaults (HBASE-2611, HBASE-12529). New or Updated Open Source Components Apache Spark 1.2 Hue 3.7 Impala 2.1 Other notables: Oracle JDK 1.8 is now supported, Impala now does incremental computation of table and column statistics (IMPALA-1122), and Apache Avro has new date, time, timestamp, and duration binary types (AVRO-739).   Over the next few weeks, we'll publish blog posts that cover some of these features in detail. In the meantime: Download Cloudera Enterprise 5.3 Explore documentation As always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs via issues.cloudera.org. ... View more

Hi Cloudera Director users,   Cloudera Director 1.0.2 was released today as a maintenance release. This is in line with our goal to quickly address bugs and enable our users to deploy production ready clusters in cloud environments such as AWS.   You can download the latest artifact here. Also, AWS QuickStart is a great way to get started with Director if you have an AWS account.   Release notes: Increased the number of inodes per partition for ephemeral drives. Director previously optimized for a small number of large files, which wasn't suitable for smaller sized instance types. The new defaults allow for a large number of files with any sized instance type.  Added support for the new AWS Frankfurt region.  Granular status updates while terminating deployments and clusters. We welcome your feedback!    - The Cloudera Director team ... View more

Hello CDH and Impala Users,   We are pleased to announce the release of the following new versions of our drivers: Impala ODBC v2.5.22 Hive ODBC v2.5.13 Impala JDBC v2.5.14 These drivers contain updates that make them compatible with CDH 5.2 and include the following features: Hive 0.13 Support Impala 2.0 Support VARCHAR, CHAR support Impala and Hive protocol update for greater performance (HIVE-3746) Mac installer improvements These drivers work for previous versions of HiveServer2 in CDH 4.1 or higher and Impala 1.0 or higher. Previous drivers continue to work on latest versions as well.   Getting started with the Cloudera Drivers: Read the Cloudera ODBC 2.5 Driver for Impala release notes and installation guide Read the Cloudera ODBC 2.5 Driver for Apache Hive release notes and  installation guide Read the Cloudera JDBC 2.5 for Impala release notes and installation guide Download the connector from the Cloudera Connectors page As always, we are happy to hear your feedback. Please send your comments and suggestions to cdh-user@cloudera.org or post to our new Community Forums. ... View more

Dear CDH, Cloudera Manager, Impala and Cloudera Navigator users, We are pleased to announce the release of Cloudera Enterprise 5.1.4, Cloudera Enterprise 5.0.5, Cloudera Navigator 2.0.3,  CDH 4.7.1, Cloudera Manager 4.8.5 and Impala 1.4.3 and 2.0.1 for CDH 4. The focus of these releases is to fix the POODLE vulnerability in SSL that was discovered in September. In addition, Cloudera Enterprise 5.1.4 contains some critical bug fixes including: Fix for duplicate actions when using the CRON syntax in Apache Oozie Kerberos ticket renewal fixes in Apache Sentry Fix for Spark to work with YARN HA Fixes in Hive to handle white spaces, delimiters, escape sequences, and delegation token cancellation Cloudera Navigator 2.0.3 includes Fix an issue with MR and HDFS extractions where solr query OR clause has too many boolean clauses. We encourage you to try it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/ content/support/en/downloads. html View the documentation: CDH 4 Release Notes Cloudera Manager 4 Release notes CDH 5 Release Notes Cloudera Manager 5 Release Notes Cloudera Navigator Release Notes Cloudera Documentation   As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums.  You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

Dear CDH, Cloudera Manager, Impala and Cloudera Navigator users,   We are pleased to announce the release of Cloudera Enterprise 5.2.1 (CDH 5.2.1, Cloudera Manager 5.2.1, and Cloudera Navigator 2.1.1)   This release is focused on fixing key bugs and includes the following:   CDH Fixes Oozie: Using cron-like syntax for Coordinator frequencies could result in duplicate actions in certain cases; this is now fixed (OOZIE-2063) YARN: Handle app-recovery failures gracefully (YARN-2010) Impala: Memory leak with string functions (IMPALA-1397) Impala: IllegalStateException when inserting results of a window function (IMPALA-1400) Impala: Read errors with Parquet files (IMPALA-1401) Impala: Regex functions don’t accept shorthand such as \d (IMPALA-1410) Impala: Queries fail with metastore exception after upgrade and compute stats (IMPALA-1416) Impala: Crashes due to bug in ClientCacheHelper (IMPALA-1445) Cloudera Manager Fixed metric collection for CDH 5.0 HDFS daemons. Fixed OutOfMemory crashes on Thrift servers in Reports Manager and Event Server. Replication commands respects JAVA_HOME if an override has been provided for it. Fixed ZooKeeper connection leaks from HBase clients used by the Service Monitor. For parcel-based installations, user home directories are created with umask 022 (instead of the user add default of 077) A new health check has been added to indicate if HDFS rolling upgrade has not been finalized. Cloudera Navigator LDAP lookups in Active Directory to resolve group membership are now working. Dropping a hive table and creating a view with same name or vice versa no longer raises an error. HDFS extraction now works after upgrading CDH from 5.1 to 5.2 Setting a property in the Hue advanced configuration snippet no longer throws a "too many Boolean clauses" error in Navigator Metadata We look forward to you trying it out using the information below: Download Cloudera Enterprise from: http://www.cloudera.com/ content/support/en/downloads. html View the documentation: CDH 5 Release Notes Cloudera Manager Release Notes Cloudera Navigator Release Notes Cloudera Documentation As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org. ... View more

We're pleased to announce the release of Cloudera Enterprise 5.2 (comprising CDH 5.2, Cloudera Manager 5.2, Cloudera Director 1.0, and Cloudera Navigator 2.1).   This release reflects our continuing investments in Cloudera Enterprise's main focus areas, including security, integration with the partner ecosystem, and support for the latest innovations in the open source platform (including Impala 2.0, its most significant release yet, and Apache Hive 0.13.1). It also includes a new product, Cloudera Director, that streamlines deployment and management of enterprise-grade Hadoop clusters in cloud environments; new component releases for building real-time applications; and new support for significant partner technologies like EMC Isilon. Furthermore, this release ships the first results of joint engineering with Intel, including WITH GRANT OPTION for Hive and Impala and performance optimizations for MapReduce.   Here are some of the highlights (incomplete; see the respective Release Notes for CDH, Cloudera Manager, and Cloudera Navigator for full lists of features and fixes):   Security Via Apache Sentry (incubating) 1.4, GRANT and REVOKE statements in Impala and Hive can now include WITH GRANT OPTION, for delegation of granting and revoking privileges (joint work with Intel under Project Rhino). Hue has a new Sentry UI that supports policy management for visually creating/editing roles in Sentry and permissions on Files in HDFS . Kerberos authentication is now supported in Apache Accumulo. Impala, authentication can now be done through a combination of Kerberos and LDAP. Data Management and Governance Cloudera Navigator 2.1 features a brand new auditing UI that is unified with lineage and discovery, so you now have access to all Navigator functionality from a single interface. Navigator 2.1 includes role-based access control so you can restrict access to auditing, metadata and policy management capabilities We’re also shipping a beta policy engine in Navigator 2.1. Targeted to GA by year-end, the policy engine allows you to set up rules and notifications so you can classify data as it arrives and integrate with data preparation and profiling tools. Try it out and let us know what you think! And we’ve added lots of top-requested enhancements, such as Sentry auditing for Impala and integration with Hue. Cloud Deployment Cloudera Director is a simple and reliable way to deploy, scale, and manage Hadoop in the cloud (initially for AWS) in an enterprise-grade fashion. It’s free to download and use, and supported by default for Cloudera Enterprise customers. Features include: Simple UI for self-service cluster spin up/teardown Dynamic scaling for spiky workloads Simple cloning of clusters Cloud blueprints for repeatable deployments Third-party software deployment within same workflow Support for custom, workload-specific deployments Support for complex cluster topologies Minimum size cluster when capacity constrained Multi-cluster dashboard Instance tracking for account billing Real-Time Architecture   Rebase on Apache HBase 0.98.6 Cell-level ACLs for fine-grained access control of data in HBase now supported Backported improvements to get and put request scheduling and throttling that provide basic QoS for multi-tenant HBase tables and clusters. Lets some production and real-time workloads take priority over ad hoc and analytic jobs. Backported patches that make Offheap Block Cache (aka bucket cache) production-ready. Now you can use large amounts of memory for read caching without the GC penalties of the past. Bucket cache is now the default. Backported authentication of clients accessing HBase via the HBase Thrift Proxy. Rebase on Apache Spark/Streaming 1.1 Rebase on Impala 2.0 Cloudera Search now provides Spark-indexing - iterative, fast index design distributed pivot facets ability to expire documents node fail recovery support for deep paging and for multithreaded faceting Apache Sqoop now supports import into Apache Parquet (incubating) file format Apache Kafka integration with CDH is now incubating in Cloudera Labs; a Kafka-Cloudera Labs parcel (unsupported) is available for installation. Integration with Flume via special Source and Sink have been provided. Impala 2.0   Disk-based query processing: enables large queries to "spill to disk" if their in-memory structures are larger than the currently available memory. (Note that this feature only uses disk for the portion that doesn't fit in the available memory.) Greater SQL compatibility: SQL 2003 analytic (window) functions, support for legacy data types (such as CHAR and VARCHAR), better compliance with SQL standards (WHERE, EXISTS, IN), and additional vendor-specific SQL extensions. Impala 2.0 is now also available for CDH 4. New Open Source Releases and Certifications   Cloudera Enterprise 5.2 includes multiple new component releases:   Apache Avro 1.7.6 Apache Crunch 0.11 Apache Hadoop 2.5 Apache HBase 0.98.6 Apache Hive 0.13.1 Apache Parquet (incubating) 1.5 / Parquet-format 2.1.0 Apache Sentry (incubating) 1.4 Apache Spark 1.1 Apache Sqoop 1.4.5 Impala 2.0 Kite SDK 0.15.0 ...with new certifications on:   Filesystems: EMC Isilon OSs: Ubuntu 14.04 (Trusty) Java: Oracle JDK1.7.0_67   Over the next few weeks, we’ll publish blog posts that cover some of these and other new features in detail. In the meantime: Download Cloudera Enterprise 5.2 Explore documentation As always, we value your feedback; please provide any comments and suggestions through our community forums. You can also file bugs via issues.cloudera.org. ... View more

We're pleased to announce the release of Kite SDK 0.17.0. This release updates the examples to CDH 5, defaults Parquet to the non-durable mode from 0.14 and prior, adds support for namespaces, and adds a kite-minicluster for easier development and integration testing against single-node Hadoop deployments. For more details see the release notes and the documentation. ... View more