Member since
09-08-2017
6
Posts
1
Kudos Received
0
Solutions
02-28-2018
10:22 AM
1 Kudo
Thank you David for the reply. We were able to get a working solution using nsupdate. Within the first boot section of the instance AMI we are setting the instance hostname to a custome name that includes the last 2 sections of the IP address (note that this needs to be the full domain name including company.com for cluster to build properly), and then doing a nsupdate to enter the hostname into the DNS. Both A and PTR records need to be entered for each instance. Within the AWS VPC we have the DHCP Options set to our company DNS server and DNS Resolution and DNS Hostname both set to NO. Cheers!
... View more
02-20-2018
04:53 PM
I am using cloudera director 2.6 in AWS to build a CM/CDH 5.13 cluster. When using the default Amazon DNS the cluster builds and executes properly - although I have no resolution back to my on-prem services. Our desire is to have the AWS instances on our corporate DNS so we built a linux AMI that is setup to resolve to our on-prem DNS, and I am using this as the AMI for our cluster instances. I have changed the settings in AWS to have DHCP options set to point to the same on-prem DNS - additionally I set the 'DNS Resolution' setting to NO. The ec2 instances created have a name ip_X_X_X_X@ourcompany.com as expected, however there is no resolution for these names. As a result of theses setting - when i use cloudera director to create a cluster it fails as it is trying to install the agent on the CM Server instance (failure to resolve the hostname). It appears that we need an automated way to add these new ec2 instances into our on-prem DNS, or is there another solution we are overlooking? Has anyone else been successful in using a similar configuration? Do you use a tool to aumoate the entry of instances into the on-prem DNS? Thanks for any insights you can provide!
... View more
Labels:
- Labels:
-
Cloudera Director
11-09-2017
02:42 AM
Yes, I have a spark gateway on the host and I copied hive-site.xml into /etc/spark/conf.
... View more
11-08-2017
09:19 AM
I am having the same issue and copying the hive-site.xml did not resolve the issue for me. I am not using spark2, but the v1.6 that comes with Cloudera 5.13 - and there is no spark/hive configuration setting. Was anyone else able to figure out how to fix this? Thanks!
... View more
11-08-2017
08:54 AM
Hello! I am seeing this same issue ... was there a resolution to your problem? Thanks.
... View more