Is there a way to see the aws command history from a Cloudera Director run? Any logs?

[#1]

I'm trying to determine the difference between the instances I've stood up for and demo and the ones Cloudera Director created. 

Is there any command history available from the Cloudera Director build?

If there is another way, perhaps querying and comparing the two environments using the aws cli, that would also work if

I have to try to reverse engineer things.

Thanks,

Kirk

[#2]

Hi,
You can check the Cloudera Director logs to see what AWS actions are taken.
If you're using the Cloudera Director client, the logs can be found at
$HOME/.cloudera-director/logs/application.log. If using the Cloudera
Director server, the logs can be found at
/var/log/cloudera-director-server/application.log.

If you let us know what you typically do to setup EC2 instances, we can
also tell you whether Director does the same or not.

-Vinithra

[#3]

Hi,

Two questions:

1.  The application log doesn't reveal the aws CLI commands that are run.  Is there a way to 

see those commands?  Either before (as a dry run feature) or after?

2. The nodes that get provisioned do not have anything in the /etc/hosts file.  How are they resolving each other?  

Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

Thanks - Kirk

[#4 · Accepted solution]

Hi Kirk - 

> 1.  The application log doesn't reveal the aws CLI commands that are run.  Is there a way to 

> see those commands?  Either before (as a dry run feature) or after? 

Cloudera Director integrates with the AWS API using the Java SDK. There is no straightforward way to translate those method calls as AWS CLI commands you can run standalone. In theory you should be able to figure out how to use the AWS CLI to start similar instances by inspecting the existing ones. 

> 2. The nodes that get provisioned do not have anything in the /etc/hosts file.  How are they resolving each other?  

> Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

We expect DNS configuration to be handled outside of Cloudera Director using different mechanisms:

1. VPC configuration to use AmazonProvidedDNS via DHCP Options Sets
2. custom bootstrap script that Director runs to configure the hostname
3. scripts embedded into a custom AMI that configure the proper hostname
   during boot and handle DNS registration

Cloudera Director doesn't attempt to perform any DNS configs.