Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Solved Go to solution

Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Explorer

I'm trying to determine the difference between the instances I've stood up for and demo and the ones Cloudera Director created. 

Is there any command history available from the Cloudera Director build?

 

If there is another way, perhaps querying and comparing the two environments using the aws cli, that would also work if

I have to try to reverse engineer things.

 

Thanks,

 

Kirk

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Expert Contributor

Hi Kirk - 

 

> 1.  The application log doesn't reveal the aws CLI commands that are run.  Is there a way to 

> see those commands?  Either before (as a dry run feature) or after? 

 

Cloudera Director integrates with the AWS API using the Java SDK. There is no straightforward way to translate those method calls as AWS CLI commands you can run standalone. In theory you should be able to figure out how to use the AWS CLI to start similar instances by inspecting the existing ones. 

 

> 2. The nodes that get provisioned do not have anything in the /etc/hosts file.  How are they resolving each other?  

> Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

 

We expect DNS configuration to be handled outside of Cloudera Director using different mechanisms:

  1. VPC configuration to use AmazonProvidedDNS via DHCP Options Sets
  2. custom bootstrap script that Director runs to configure the hostname
  3. scripts embedded into a custom AMI that configure the proper hostname
    during boot and handle DNS registration

Cloudera Director doesn't attempt to perform any DNS configs. 

3 REPLIES 3
Highlighted

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Contributor
Hi,
You can check the Cloudera Director logs to see what AWS actions are taken.
If you're using the Cloudera Director client, the logs can be found at
$HOME/.cloudera-director/logs/application.log. If using the Cloudera
Director server, the logs can be found at
/var/log/cloudera-director-server/application.log.

If you let us know what you typically do to setup EC2 instances, we can
also tell you whether Director does the same or not.

-Vinithra

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Explorer

Hi,

 

Two questions:

 

1.  The application log doesn't reveal the aws CLI commands that are run.  Is there a way to 

see those commands?  Either before (as a dry run feature) or after?

 

2. The nodes that get provisioned do not have anything in the /etc/hosts file.  How are they resolving each other?  

Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

 

Thanks - Kirk

Re: Is there a way to see the aws command history from a Cloudera Director run? Any logs?

Expert Contributor

Hi Kirk - 

 

> 1.  The application log doesn't reveal the aws CLI commands that are run.  Is there a way to 

> see those commands?  Either before (as a dry run feature) or after? 

 

Cloudera Director integrates with the AWS API using the Java SDK. There is no straightforward way to translate those method calls as AWS CLI commands you can run standalone. In theory you should be able to figure out how to use the AWS CLI to start similar instances by inspecting the existing ones. 

 

> 2. The nodes that get provisioned do not have anything in the /etc/hosts file.  How are they resolving each other?  

> Typically, the hosts file configuration is critical to getting Cloudera Manager to run successfully.

 

We expect DNS configuration to be handled outside of Cloudera Director using different mechanisms:

  1. VPC configuration to use AmazonProvidedDNS via DHCP Options Sets
  2. custom bootstrap script that Director runs to configure the hostname
  3. scripts embedded into a custom AMI that configure the proper hostname
    during boot and handle DNS registration

Cloudera Director doesn't attempt to perform any DNS configs. 

Don't have an account?
Coming from Hortonworks? Activate your account here