Member since
06-26-2013
354
Posts
68
Kudos Received
27
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3484 | 08-05-2016 10:36 AM | |
5584 | 06-02-2016 04:57 PM | |
5657 | 05-31-2016 03:47 PM | |
4778 | 04-11-2016 11:26 AM | |
10253 | 03-07-2016 02:04 PM |
07-21-2014
10:48 AM
Cloudera has two products, Cloudera Express and Cloudera Enterprise:
Cloudera Express (FREE): includes the open source platform, CDH, and Cloudera Manager (cluster/system management software). No support.
Cloudera Enterprise (PAID): Includes CDH, Cloudera Manager + extended enterprise features, and support & indemnity. Pricing is based on for what components you want support (see editions here).
You can also choose to use CDH without Cloudera Manager, but you'll save yourself a lot of time and work if you do use the latter.
I hope that helps!
... View more
07-18-2014
11:29 AM
Cloudera is pleased to announce support for Accumulo 1.6.0 on both CDH 4 and CDH 5.
As of version 5.1.0, Cloudera Manager now ships with support to deploy and manage Accumulo 1.6.0 on CDH 4.6.0 or later and CDH 5.1.0 or later.
* Docs: http://tiny.cloudera.com/accumulo-docs
* CM 5.1.0 Download: http://tiny.cloudera.com/cm-5.1.0
* Accumulo 1.6.0 on CDH 5 (CDH5.1.0 or later): http://archive.cloudera.com/accumulo-c5/
* Accumulo 1.6.0 on CDH 4 (CDH4.6.0 or later): http://archive.cloudera.com/accumulo/
... View more
07-17-2014
02:30 PM
Dear CDH, Cloudera Manager, Impala and Search Users,
We are thrilled to announce the GA release of Cloudera Enterprise 5.1 (CDH 5.1 and Cloudera Manager 5.1), Cloudera Navigator 2.0, Impala 1.4 for CDH 4, Impala ODBC v2.5.17 and Hive ODBC v2.5.10.
Cloudera Enterprise 5.1
Cloudera Enterprise 5.1 contains a number of new features and component versions including the ones below:
Apache Hadoop
HDFS
Extended ACL Support
NFS Enhancements
HTTPS support
YARN
YARN + Impala GA: CDH 5.1 now supports three modes of Resource Management:
Static Partitioning (CDH 4.3+)
Admission Control (CDH 5.0+)
Dynamic Prioritization (CDH 5.1+)
As part of Dynamic Prioritization, Llama now supports HA
Configuration similar to other HA elements (Active-Passive)
Apache HBase
HBase 0.98.1 rebase
Improved WAL write performance
Reverse scans
MapReduce over snapshots
Apache Spark
Rebase to Spark 1.0
Spark Streaming integration with Kerberos
Application History Server improves monitoring capabilities
Sparse Vector Support in MLLib
Improvements to Avro integration
Simplified job submission to YARN cluster
Security: Authentication of all Spark communications
Apache Sentry
Rebase to 1.3
Support for Grant/Revoke statements via beeline (HiveServer2)
Policy files no longer required
Sentry fixes: job tracking in Hue, HDFS permissions inheritance
Impala
Release of Impala 1.4.0
YARN-integrated Resource Management is GA
DECIMAL across Hive, Impala, Avro, Parquet, and text file formats
Integration with HDFS Caching
Fixed folder inheritance issues for better framework interoperability (IMPALA-827)
LDAPS Support for more secure AD username/password authentication
ORDER BY without LIMIT
Performance improvements for COMPUTE STATS and queries
DDL support for creating a new table from an existing Parquet file schema
DDL support for creating Avro tables
Additional built-in functions like TRUNC, EXTRACT, and stats functions
Apache Hive
DECIMAL across Hive, Impala, Avro, Parquet, and text file format
Fixed folder inheritance issues for better framework interoperability (HIVE-6892)
Cloudera Search
Document Level Security
Parquet Support
Search QuickStart Document and Script
HBase Lily Indexer update
Apache Oozie
Added ability to submit Sqoop jobs from Oozie CLI (Knox competitive)
Completed testing of JMS/Email notification for workflow and Coordinator action status changes
Completed testing of SLA notifications
Replaced/Improved Metrics instrumentation (for Cloudera Manager monitoring)
LAST_ONLY execution mode now works correctly
Cloudera Manager
Fine grain access for Cloudera Manager
Two additional roles in Cloudera Manager - Operator and Configurator
Impala Llama HA support
Security Updates
Integrate directly with Active Directory to set up kerberos principals needed for CDH daemons
New wizard to add kerberos to an existing unsecure cluster
Manage and deploy kerberos client configurations (krb5.conf).
Sentry support for Grant/Revoke
Support for SSL
Manage Hadoop SSL related configurations
Monitor Hadoop services when SSL is enabled
Monitoring
Updates to Oozie monitoring
New Hive Metastore Canary
Hue
Rebase to 3.6
Search App v2
100% dynamic dashboards
Drag and drop dashboard builder
Text, Timeline, Pie, Line, Bar, Map, Filters, Grid, and HTML Widgets
Solr Index Designer (basic up-and-running wizard, from a file in HDFS)
View support for Snappy, Avro, Parquet files
Multi NT domain
LDAP Nested Groups
Impala HA
Close commands for Impala/Hive sessions and queries
Other Rebases
Apache Mahout - 0.9
Apache Crunch - 0.10
Flume 1.5.0
We look forward to you trying it out using the information below:
Download Cloudera Enterprise from: cloudera.com/downloads
View the documentation:
CDH 5 Documentation
Cloudera Manager 5 Documentation
As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org.
Cloudera Navigator 2.0
Cloudera Navigator is a fully-integrated governance, compliance, and risk management solution for Hadoop. Cloudera Navigator provides comprehensive metadata, lineage, and auditing support across the enterprise data hub.
Unified technical and business metadata
Consolidate technical metadata for Hadoop files and tables, plus add custom tags and comments so that you can easily track, annotate and classify data in alignment with business rules.
Lineage
View upstream and downstream column-level lineage in an easy-to-follow graph so that you can quickly identify the origin of a data set and its impact on downstream analysis.
Supports Hive, Pig, Sqoop 1, MapReduce, YARN, Oozie
Auditing
View and summarize all data access attempts with a simple, queryable interface so you can quickly identify outliers and security breaches
Supports Hive, Impala, HBase, HDFS, Sentry
Security
Cloudera Navigator includes Navigator Encrypt and Navigator Key Trustee, formerly known as Gazzang zNcrypt and Gazzang zTrustee. These products provide enterprise-grade encryption and key management.
Links
Product Information
http://www.cloudera.com/content/cloudera/en/products-and-services/cloudera-enterprise/cloudera-navigator.html
Documentation: http://www.cloudera.com/content/support/en/documentation/cloudera-navigator/cloudera-navigator-v2-latest.html
Impala 1.4 on CDH4
Impala 1.4 for CDH4 contains features that are included in Cloudera Enterprise 5.1 which are supported by CDH4:
Fixed folder inheritance for Sentry (IMPALA-827)
LDAPS Support for more secure AD username/password authentication
ORDER BY without LIMIT
DDL support for creating a new table from an existing Parquet file schema
DDL support for creating Avro tables
Fixed folder inheritance issues for better framework interoperability (IMPALA-827)
Performance improvements for COMPUTE STATS and queries
Additional built-in functions like TRUNC, EXTRACT, and stats functions
Note that Impala 1.4 does not work with Cloudera Manager 4.7 or earlier. If your environment is managed by Cloudera Manager 4.7 or earlier, you should upgrade to Impala 1.4 and Cloudera Manager 4.8 together.
For more details on new Impala features, please see “New Features in Impala” for CDH 4: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Cloudera-Impala-Release-Notes/cirn_new_features.html
“Installing and Using Cloudera Impala” guide for CDH 4: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Impala/latest/Installing-and-Using-Impala/Installing-and-Using-Impala.html
As always, we are happy to hear your feedback. Please send your comments and suggestions to impal...@cloudera.org or through our new community forums. You can also file bugs in the Impala project at issues.cloudera.org
Impala ODBC 2.5.17 and Hive ODBC 2.5.10
The new ODBC drivers include the following new functionality:
Impala ODBC fixes the default fetch size for faster client returns
Improved Mac OSX client installation
We look forward to you trying it out using the links below:
Impala ODBC v2.5.17
Hive ODBC v2.5.10
... View more
07-15-2014
04:19 PM
Version 0.15.0 contains the following notable changes:
Kite artifacts are built against Apache Hadoop 2 and related projects, and are now available in Maven Central.
Added new introduction and concepts documentation.
Added a new Datasets convenience class for opening and working with Datasets, superseding DatasetRepositories.
Deprecated partition related methods in Dataset in favor of the views API.
Added a CLI copy task for copying datasets and also for dataset format conversion and data compaction.
Added an application parent POM that makes it easy to use Kite in a Maven project. The examples now use this parent POM.
Updated to Crunch 0.10.0
Morphlines Library
Added morphline command that parses an InputStream that contains protobuf data: readProtobuf (Rober Fiser via whoschek)
Added morphline command that extracts specific values from a protobuf object, akin to a simple form of XPath: extractProtobufPaths (Rober Fiser via whoschek)
Added morphline command that removes all record fields for which the field name matches a blacklist but not a whitelist: removeFields
Added optional parameters maxCharactersPerRecord and onMaxCharactersPerRecord to morphline command readCSV
Upgraded kite-morphlines-maxmind module from maxmind-db-0.3.1 to bug fix release maxmind-db-0.3.3
Upgraded kite-morphlines-core module from metrics-0.3.1 to bug fix release metrics-0.3.2
The full change log is available from JIRA.
... View more
07-10-2014
02:38 PM
Dear CDH and Cloudera Manager users,
We are pleased to announce the release of Cloudera Enterprise 5.0.3 (CDH 5.0.3 and Cloudera Manager 5.0.2).
This release is focused on fixing key bugs and includes the following:
CDH Fixes
FLUME-2245 - HDFS files with errors unable to close
FLUME-2416 - Use CodecPool in compressed stream to prevent leak of direct buffers
HBASE-10871 Indefinite OPEN/CLOSE wait on busy RegionServers
HDFS-5891 - WebHDFS should not try connecting the DataNode during redirection
HDFS-6021 - NPE in FSImageFormatProtobuf upgrading from layout -52 to -53
HDFS-6077 - running slive with WebHDFS on secure HA cluster fails with unknown host exception
HDFS-6340 - DataNode can't finalize upgrade
HDFS-6475 - WebHDFS clients fail without retry because incorrect handling of StandbyException
HDFS-6510 - WebHDFS clients clear the delegation token on retry (for HA), thus failing retry requests
HDFS-6527 - Edit log corruption due to deferred INode removal
HDFS-6563 - NameNode cannot save fsimage in certain circumstances when snapshots are in use
HUE-1928 - [beeswax] HiveServer2 supports pass-through LDAP authentication
HUE-2085 - [core] Update migration dependencies
HUE-2184 - [core] Connect to Oracle via Service Name
HUE-2192 - [core] Create parameter to choose LDAP username for HiveServer2
HUE-2193 - [beeswax] HiveServer2 pass-through LDAP authentication at thrift level
OOZIE-1621 - Add proper error code and error message for sharelib exceptions.
OOZIE-1890 - Make oozie-site empty and reconcile defaults between oozie-default and the code
OOZIE-1907 - DB upgrade from 3.3.0 to trunk fails on derby
SOLR-5593 - shard leader loss due to ZK session expiry
SOLR-5915 - Cannot set parserImpl=... with PreAnalyzedField
SOLR-6161 - OutOfMemoryError Not Thrown in sendError
YARN-1550 - NPE in FairSchedulerAppsBlock#render
YARN-2155 - FairScheduler: Incorrect threshold check for preemption
We look forward to you trying it out using the information below:
Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html
View the documentation:
CDH 5 Release Notes
CDH 5 Documentation
Cloudera Manager Release Notes
Cloudera Manager 5 Documentation
As always, we are happy to hear your feedback in these forums. You can also file bugs through our external jira projects on issues.cloudera.org.
... View more
07-09-2014
01:49 PM
Daan,
I think you'd need to ask Amazon about this; it provides support for Impala on EMR.
... View more
06-12-2014
02:36 PM
Dear CDH and Cloudera Manager users,
We are pleased to announce the release of Cloudera Enterprise 5.0.2 (CDH 5.0.2 and Cloudera Manager 5.0.2). This release is focused on fixing key bugs and includes the following:
Cloudera Manager
Cloudera Manager Impala Query Monitoring works with Impala 1.3.1
CDH Fixes
HADOOP-10556: Add toLowerCase support to auth_to_local rules for service name
HADOOP-10638: Updating hadoop-daemon.sh to work as expected when nfs is started as a privileged user.
HADOOP-10639: FileBasedKeyStoresFactory initialization is not using default for SSL_REQUIRE_CLIENT_CERT_KEY
HADOOP-10658: SSLFactory expects truststores being configured
HBASE-6990: Pretty print TTL
HBASE-10312: Flooding the cluster with administrative actions leads to collapse
HBASE-10371: Compaction creates empty hfile, then selects this file for compaction and creates empty hfile and over again
HDFS-6326: WebHdfs ACL compatibility is broken
HIVE-6913: Hive unable to find the hashtable file during complex multi-staged map join
HIVE-5380: Non-default OI constructors should be supported for backwards compatibility
PIG-3677: ConfigurationUtil.getLocalFSProperties can return an inconsistent property set
YARN-2073: Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free
We look forward to you trying it out using the information below:
Download Cloudera Enterprise from: http://www.cloudera.com/content/support/en/downloads.html
View the documentation:
CDH 5 Release Notes
CDH 5 Documentation
Cloudera Manager Release Notes
Cloudera Manager 5 Documentation
As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org.
Cloudera Team
... View more
05-30-2014
09:16 AM
Dear CDH, Cloudera Manager, and Search Users,
We are pleased to announce the release of CDH 4.7 and Search 1.3. These are both minor releases that contain multiple bug fixes.
CDH 4.7
Bug fixes in CDH 4.7 include:
HBase
HBASE-10514 - Forward port HBASE-10466, possible data loss when failed flushes
HBASE-10257 - Master aborts due to assignment race
HBASE-8912 - AssignmentManager throws IllegalStateException from PENDING_OPEN to OFFLINE
HDFS
HDFS-6289 - HA failover can fail if there are pending DataNode messages for DataNodes which no longer exist
HDFS-5944 - LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ right and cause SecondaryNameNode failed do checkpoint
HDFS-6191 - Disable quota checks when replaying edit log
HDFS-4943 - WebHdfsFileSystem does not work when original file path has encoded chars
HDFS-5064 - Standby checkpoints should not block concurrent readers
HDFS-6160 - TestSafeMode occasionally fails
HDFS-5496 - Make replication queue initialization asynchronous
HDFS-5438 - Flaws in block report processing can cause data loss
HDFS-5255 - Distcp job fails with hsftp when https is enabled in insecure cluster
HDFS-5074 - Allow starting up from an fsimage checkpoint in the middle of a segment
HDFS-4879 - Add "blocked ArrayList" collection to avoid CMS full GCs
Hadoop
HADOOP-9454 - Support multipart uploads for s3native
MapReduce
MAPREDUCE-5877 - Inconsistency between JobTracker/TaskTracker for tasks taking a long time to launch
Hue
HUE-1962 - Support int row key from Hive table
HUE-2060 - LDAP import commands carry incorrect import statements
HUE-1992 - Username to lowercase switch for RemoteUserDjangoBackend
HUE-1873 - Result data not HTML encoded
HUE-1897 - workflow IDs have double trailing slashes
Hive
HIVE-6005 - BETWEEN is broken after using KRYO
HIVE-4222 - Timestamp type constants cannot be deserialized in JDK 1.6 or earlier
HIVE-5380 - Non-default OI constructors should be supported for backwards compatibility
HIVE-5263 - Query Plan cloning time could be improved by using Kryo
Oozie
OOZIE-1699 - Some of the commands submitted to Oozie internal queue are never executed
Flume
FLUME-2357 - HDFS sink should retry closing files that previously had close errors
For more details on new features, see the release notes: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Release-Notes/CDH4-Release-Notes.html
Refer to the “CDH4 Quick Start Guide” or the “CDH4 Installation Guide” for more information on how to install this update release.
CDH4 Quick Start Guide: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Quick-Start/CDH4-Quick-Start.html
CDH4 Installation Guide: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/CDH4-Installation-Guide.html
As always, we welcome your feedback. Please send your comments and suggestions to cdh-user@cloudera.org or through our community forums. You can also file bugs through the Distro project on issues.cloudera.org.
Search 1.3
Cloudera Search 1.3.0 is an update to the Cloudera Search product. This release contains several bug fixes. Note that Search 1.3.0 requires CDH4.7.
For more details on new features, see “New Features in Cloudera Search Version 1.3.0”: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Search/latest/Cloudera-Search-Release-Notes/Cloudera-Search-Release-Notes.html
For access to the Cloudera Search Documentation: http://www.cloudera.com/content/support/en/documentation/cloudera-search/cloudera-search-documentation-v1-latest.html
As always, we welcome your feedback. Please send your comments and suggestions to search-user@cloudera.org or through our new community forums. You can also file a bug using the Search component in the Distro project on issues.cloudera.org
... View more
05-22-2014
09:29 AM
1 Kudo
ATP,
Briefly: Cloudera Enterprise subscriptions are available in 3 editions: Basic, Flex, and Enterprise Data Hub.
- Basic includes support for "core" Hadoop ecosystem (Core, Hive, Pig, Oozie, etc)
- Flex is Basic + support for one "premium" component per cluster (Spark, HBase, Accumulo, Search, Impala, Navigator)
- Data Hub Edition is all you can eat
- All three include Cloudera Manager with enterprise features
See matrix here:
http://www.cloudera.com/content/cloudera/en/products-and-services/product-comparison.html
... View more
05-14-2014
08:38 AM
Dear CDH, Cloudera Manager, and Cloudera Impala users,
We are pleased to announce the release of Cloudera Enterprise 5.0.1 (CDH 5.0.1 and Cloudera Manager 5.0.1) and Cloudera Manager 4.8.3.
Cloudera Enterprise 5.0.1
This release is focused on fixing key bugs and includes the following:
Component updates
Impala 1.3.1
Cloudera Manager
HDFS NFS gateway does not work on all Cloudera-supported platforms
Replace YARN_HOME with HADOOP_YARN_HOME during upgrade
Insufficient password hashing in Cloudera Manager
Upgrade to Cloudera Manager 5.0.0 from SLES older than Service Pack 3 with PostgreSQL older than 8.4 fails
MR1 to MR2 import fails on a secure cluster
After upgrade from CDH 4 to CDH 5, Oozie is missing workflow extension schemas
CDH Fixes
SOLR-5608 - Frequently reproducible failures in CollectionsAPIDistributedZkTest#testDistribSearch
HIVE-6648 - Fixed permission inheritance for multi-partitioned tables
HIVE-6740 - Fixed addition of Avro jars to the classpath
HIVE-6575 - select * fails on parquet table with map datatype
OOZIE-1794 - java-opts and java-opt in the Java action don't always work properly in YARN
HADOOP-10456 - Bug in Configuration.java exposed by Spark (ConcurrentModificationException)
HUE-2061 - Task logs are not retrieved if containers not on the same host
HADOOP-10442 - Group look-up can cause segmentation fault when certain JNI-based mapping module is used.
HDFS-5064 - Standby checkpoints should not block concurrent readers
HDFS-6094 - The same block can be counted twice towards safe mode threshold
HDFS-6231 - DFSClient hangs infinitely if using hedged reads and all eligible datanodes die
We look forward to you trying it out using the information below:
Download Cloudera Enterprise from: cloudera.com/downloads
View the documentation:
CDH5 Release Notes
CDH 5 Documentation
Cloudera Manager Release Notes
Cloudera Manager 5 Documentation
As always, we are happy to hear your feedback. Please send your comments and suggestions to the user group or through our community forums. You can also file bugs through our external jira projects on issues.cloudera.org.
Cloudera Manager 4.8.3
Cloudera Manager 4.8.3 is a maintenance release and has a number of bug-fixes and usability improvements.
For more details, see the release notes: http://www.cloudera.com/content/cloudera-content/cloudera-docs/Cloudera Manager4Ent/latest/Cloudera-Manager-Release-Notes/Cloudera-Manager-Release-Notes.html
All of the Cloudera Manager documentation can be found here:
http://www.cloudera.com/content/support/en/documentation/manager/cloudera-manager-v4-latest.html
You can download this latest Cloudera Manager here or you can upgrade from an existing installation of Cloudera Manager. Installation and upgrade documentation can be found here.
As always, we are happy to hear your feedback. Please send your comments and suggestions to scm-user@cloudera.org or through our new community forums.
... View more