Abstract: CDP is using many managed cloud services from different CSPs as underlying infrastructure. The DNS configuration in a typical CDP onboarding is critical to a successful deployment. This article introduces some common DNS setup issues with AWS and corresponding solutions.
AWS services with private endpoints used by CDP
Most enterprise AWS consumers are using HUB-SPOKE network architecture with custom private DNS. The Custom private DNS usually resides on HUB VPC. Correct DNS configuration needs to be created for a successful deployment.
AWS managed services CDP is using
- RDS
- Elastic Kubernetes Service
- Elastic File Service
In a typical HUB-Spoke network, the custom private DNS server is usually deployed in the HUB network. There are several other DNS servers that are being used.
- AWS Public DNS: Hosted by AWS on internet
- AWS Default Private DNS: One per VPC. At 169.254.169.253
Different AWS services use different DNS for its domain registration.
- AWS RDS create an endpoint in the CDP VPC, and the DNS record for this endpoint is registered in AWS Public DNS
- AWS EKS register an endpoint in the CDP VPC, and the DNS record for this endpoint is registered in AWS Public DNS
- AWS EFS create an endpoint in the CDP VPC, and the DNS record for this endpoint is registered in AWS VPC Default DNS
Challenges introduced to CDP deployment and resolution
When using AWS VPC default DNS for CDP VPC
The VPC default DNS can return all the resolves for RDS, EKS, and EFS endpoints.
When using custom private DNS in the same VPC as CDP resources
Recursive lookup for RDS and EKS is DNS native feature, and custom private DNS will recursively lookup the DNS record and come back from AWS public DNS. But because the EFS DNS record is in VPC Default DNS, it can’t be recursively looked up. Users will need to manually configure conditional forward from the custom private DNS to the EFS endpoint.
Recursive lookup for RDS and EKS is DNS native feature, and custom private DNS will recursively lookup the DNS record and come back from AWS public DNS. But because the EFS DNS record is in VPC Default DNS, it can’t be recursively looked up.
Custom DNS on another network cannot do conditional forward to the AWS VPC default DNS for CDP VPC.
Users will have to create an AWS Route 53 Resolver with an inbound resolve endpoint, so that the custom private DNS can forward the conditional forward for EFS to this Resolver inbound endpoint.
So far, Data Warehouse doesn’t support custom private DNS. Which means if the customer’s network team does not allow VPC Default DNS, Data warehouse cannot be deployed.
Sometimes, we can convince the network team to use AWS VPC default DNS, but that will introduce another dilemma that the resources in CDP VPC will not be able to resolve the other resources in the custom private domain.
Users can create an AWS Route 53 resolver with outbound endpoint and forward the requests to the custom private domain to the custom private DNS server.
