question Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs? in Support Questions

Is there a way to see the aws command history from a Cloudera Director run? Any logs?

ProKirk — Fri, 10 Jul 2015 17:31:18 GMT

I'm trying to determine the difference between the instances I've stood up for and demo and the ones Cloudera Director created.

Is there any command history available from the Cloudera Director build?

If there is another way, perhaps querying and comparing the two environments using the aws cli, that would also work if

I have to try to reverse engineer things.

Thanks,

Kirk

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Vinithra — Fri, 10 Jul 2015 18:30:55 GMT

Hi,
You can check the Cloudera Director logs to see what AWS actions are taken.
If you're using the Cloudera Director client, the logs can be found at
$HOME/.cloudera-director/logs/application.log. If using the Cloudera
Director server, the logs can be found at
/var/log/cloudera-director-server/application.log.

If you let us know what you typically do to setup EC2 instances, we can
also tell you whether Director does the same or not.

-Vinithra

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

ProKirk — Mon, 13 Jul 2015 18:51:14 GMT

Hi,

Two questions:

1. The application log doesn't reveal the aws CLI commands that are run. Is there a way to

see those commands? Either before (as a dry run feature) or after?

2. The nodes that get provisioned do not have anything in the /etc/hosts file. How are they resolving each other?

Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

Thanks - Kirk

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Andrei Savu — Thu, 16 Jul 2015 02:18:13 GMT

Hi Kirk -

> 1. The application log doesn't reveal the aws CLI commands that are run. Is there a way to

> see those commands? Either before (as a dry run feature) or after?

Cloudera Director integrates with the AWS API using the Java SDK. There is no straightforward way to translate those method calls as AWS CLI commands you can run standalone. In theory you should be able to figure out how to use the AWS CLI to start similar instances by inspecting the existing ones.

> 2. The nodes that get provisioned do not have anything in the /etc/hosts file. How are they resolving each other?

> Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

We expect DNS configuration to be handled outside of Cloudera Director using different mechanisms:

VPC configuration to use AmazonProvidedDNS via DHCP Options Sets
custom bootstrap script that Director runs to configure the hostname
scripts embedded into a custom AMI that configure the proper hostname
during boot and handle DNS registration

Cloudera Director doesn't attempt to perform any DNS configs.