2206
Posts
230
Kudos Received
82
Solutions
About
My expertise is not in hadoop but rather online communities, support and social media. Interests include: photography, travel, movies and watching sports.
My Accepted Solutions
Title | Views | Posted |
---|---|---|
442 | 05-07-2025 11:41 AM | |
916 | 02-27-2025 12:49 PM | |
2792 | 06-29-2023 05:42 AM | |
2368 | 05-22-2023 07:03 AM | |
1736 | 05-22-2023 05:42 AM |
02-15-2017
05:51 AM
Welcome to the community @HPathak. I have good news and bad news. The good news is that the quickstart VM only requires 4GB+ of RAM allocated to the VM if you are chosing the CDH5 option. The bad news is that if you want to use Cloudera Manager (Cloudera Express option and above) you will need to allocate 8GB+ or RAM to the VM.
This requirement goes back as far as I can recall so if there is an older version requiring less RAM I don't think it would be suitable to your evaluation.
I hope this helps.
... View more
02-15-2017
05:31 AM
Welcome to the community @Hamid77. As stated in our Community Knowledge article on installing the QuickStart VM:
We HIGHLY recommend that you install a “Download Manager” extension to your web browser and utilize that to manage the download of the Quickstart VM as it is a VERY large file and can easily get corrupted during download due to connectivity issues.
If this doesn't solve your issue I suggest starting a new thread with a complete description of your setup and steps taken to receive the error.
... View more
02-14-2017
05:43 AM
1 Kudo
I should have clarified a little bit more on the search. When you use the search for the community it provides both forum and Community Knowledge articles that match the terms. The seperator is for if you want to look at just the knowledge articles. You can see the icons next to the search results for both kinds of content in the screen shot below.
... View more
02-14-2017
05:25 AM
Thank you for the feedback @mbigelow. This is just the kind of verification we need to back up our own thoughts on improving the community. While I cannot promise all the items on your list will come to fruition, they will be considered. For example, separated search is due to the platform that underlies the community.
... View more
02-07-2017
05:43 AM
Hi @Neelesh,
I will send you a private message for further information.
... View more
02-06-2017
05:53 AM
1 Kudo
Hi @AnisurRehman,
Did you check out the follow up posts from the series including:
Ralph Kimball and Kaiser Permanente: Q&A Part I – Hadoop and the Data Warehouse
Ralph Kimball and Kaiser Permanente: Q&A Part II – Building the Landing Zone
... View more
01-21-2017
07:42 AM
1 Kudo
I would also suggest checking out our community knowledge article How to setup Cloudera Quickstart VM. It has some great tips to a smoother install and hits on some of the common questions and issues people have. Best of luck. 🙂
... View more
01-20-2017
12:42 PM
We have just published a new Engineering blog post How to secure ‘Internet exposed’ Apache Hadoop that may be of interest as well.
... View more
01-18-2017
10:46 AM
2 Kudos
Cloudera takes cluster security very seriously, and provides guidelines for securing CDH environments:
http://www.cloudera.com/documentation/enterprise/latest/topics/sg_edh_overview.html
Security measures become especially important when clusters are exposed to the internet. For example, hackers using network scanning tools are actively looking for WebHDFS ports on clusters, and when they find an open port, they can wreak havoc on the cluster (delete data, steal data, corrupt data).
There are many other services and access considerations that should be protected as well.
It’s imperative that you design clusters in a secure fashion which will not leave the services interfaces exposed to the Internet this way. It’s our strong recommendation that you secure your cluster with kerberos, TLS, proper firewall or proxy access, and use the guidance from our security guide to protect your deployment.
Users affected:
All unsecured clusters exposed to the internet
Impact:
Cluster data may be copied, downloaded and deleted. Cluster altered or permanently disabled
Action required:
For perimeter security consider a quick test to be a check of: "Can I access this cluster from a public network with no vpn or other security in place?" If so, check with your network administration team or in the Cloudera community discussion forums as a resource to this evaluation and setup of proper security.
Securing a cluster requires the following
Perimeter security configured to protect access to your deployment
https://www.cloudera.com/documentation/enterprise/latest/topics/sg_edh_overview.html
If using Amazon or Azure, review the Director discussion on cloud security group setup and allow only inbound SSH for authentication and encryption of access for VPC security for cloud here at:
http://www.cloudera.com/documentation/director/latest/topics/director_get_started.html
A KDC to be provided to enable kerberos authentication
https://www.cloudera.com/documentation/enterprise/latest/topics/sg_auth_overview.html
Enabling kerberos authentication through Cloudera Manager's web UI by using the wizard (consider enabling Cloudera Manager TLS first)
https://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_intro_kerb.html
Enabling encryption of data in transit for both RPC and data, and Cloudera Manager as well as cluster web UI's
Encryption Overview:
http://www.cloudera.com/documentation/enterprise/latest/topics/sg_encryption.html
TLS:
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_guide_ssl_certs.html
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_config_tls_security.html
Kerberos RPC (HDFS Encrypted Transport):
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_hdfs_encrypt_transport.html
Best practice considerations like changing default passwords, creating users and groups deploying navigator auditing and reviewing access attempts on an ongoing basis.
Cloudera provides an overview on securing a cluster properly for the Cloudera 5.x platform in a Vision blog post. It is provided here for reference:
https://vision.cloudera.com/production-ready-hadoop-an-overview-of-security-in-cloudera-5/
To check if your existing cluster has authentication security enabled: Navigate within Cloudera Manager from the home page to the Administration menu. Click the "Security" sub menu. A table of the clusters being managed is presented, and the statement "Successfully enabled Kerberos" will be next to the cluster name. The following link discusses the concepts and steps to completing this setup properly:
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_authentication.html
If you are using CDH without Cloudera Manager, both the hadoop.security.authentication parameter needs to not be set to “kerberos”, and the hadoop.security.authorization parameter needs to be set to “true” in core-site.xml to indicate that security is enabled:
http://www.cloudera.com/documentation/enterprise/latest/topics/cdh_sg_hadoop_security_enable.html
To verify if TLS is enabled for Cloudera Manager and Navigator, navigate from the Cloudera Manager home page to the Administration Menu -> Settings, and search for TLS in the configuration settings search field:
http://www.cloudera.com/documentation/enterprise/latest/topics/how_to_configure_cm_tls.html
To verify if TLS is enabled for CDH components managed by Cloudera Manager, search for “tls enabled” in each of the services:
http://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_hadoop_ssl_cm.html
To verify if TLS is enabled for CDH components not managed by Cloudera Manager, look for the setting “hadoop.ssl.enabled” within the configuration files.
For CDH and hadoop community users the following Apache reference documentation can be consulted for considerations on securing webHDFS.
https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Authentication
Here is a copy of the Apache release documentation in our mirror for current platform:
https://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-hdfs/WebHDFS.html?_ga=1.208877266.1151128972.1420475180#Authentication
If your cluster has been compromised, data has been deleted, or you would like to engage with a Cloudera security professional services team member, please reach out to your account manager or contact us at sales@cloudera.com.
... View more
01-10-2017
05:22 AM
Thank you for marking your issue as solved @DanielWhite. Can you advise what the solution was?
... View more