Support Questions

Find answers, ask questions, and share your expertise

How do I control the storage allocation with Cloudera Director Client config file?

avatar
Explorer

Hi,

 

I created a cluster using Cloudera Director Client bootstrapping 5 AWS nodes.  Each node has 3 storage volumes of ~400GB  each:

 

[root@ip-10-0-0-48 data3]# df -h

Filesystem      Size  Used Avail Use% Mounted on

/dev/xvde        99G  6.1G   88G   7% /

tmpfs           7.4G  8.0K  7.4G   1% /dev/shm

/dev/xvdf       414G  199M  414G   1% /data0

/dev/xvdg       414G  199M  414G   1% /data1

/dev/xvdh       414G  199M  414G   1% /data2

/dev/xvdi       414G  199M  414G   1% /data3

 

How can I trim these from the config or resize them?  I didn't see anywhere in the config file to adjust storage beyond the EBS boot volume.

 

Thanks,

 

Kirk

1 ACCEPTED SOLUTION

avatar
Super Collaborator

The log you sent along seems to be for a different, failed run of the Director client where it couldn't find the configuration file you specified ("./demok16.aws.simple.conf"). There should have been a log generated for the successful run that bootstrapped the cluster you were talking about. It's possible that it was overwritten by a later run, or maybe you installed the client locally via tarball (if so, it's in a logs directory under where you decompressed it)?

 

The instance type m1.xlarge does come with 4 instance store (ephemeral) volumes of 420 GB each, so what you're seeing is normal for that instance type. Director preserves those volumes and doesn't currently provide a way to resize them. (Also, Director should be naming the devices /dev/sdb through /dev/sde, unless you've configured custom configuration values for lp.ec2.ephemeral.deviceNamePrefix and lp.ec2.ephemeral.rangeStart - the log would show some more information about that.)

 

If you don't need as much ephemeral storage, then you can try a different instance type - m1.xlarge is an older type and not recommended by AWS anyway.

 

  • m3.xlarge has the same vCPU count and memory as m1.xlarge (4, 15GB) but only 2 40GB SSD instance stores. It's also cheaper than m1.xlarge, according to AWS docs.
  • The new m4.xlarge has 4 vCPU, 16 GB memory, and does away with instance stores completely.

View solution in original post

3 REPLIES 3

avatar
Super Collaborator

Hi Kirk,

 

To help figure out your situation, could you please send along the application.log file that Director wrote to while creating your cluster? It should be under /var/log/cloudera-director-client. Also, it would help if you send your client configuration file containing your cluster details - just be sure to remove any passwords or AWS keys before doing so. I'm particularly interested in knowing what instance type and AMI you used for your nodes.

 

Thanks,

 

Bill

avatar
Explorer

Thanks for the response. 

Here's the config file:

 

#

# Copyright (c) 2014 Cloudera, Inc. All rights reserved.

#

 

#

# Simple AWS Cloudera Director configuration file with automatic role assignments

# that works as expected if you use a single instance type for all cluster nodes

#

 

#

# Cluster name

#

 

name: demok16

 

#

# Cloud provider configuration (credentials, region or zone and optional default image)

#

 

provider {

    type: aws

 

    #

    # Get AWS credentials from the OS environment

    # See http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html

    #

    # If specifying the access keys directly and not through variables, make sure to enclose

    # them in double quotes.

 

    accessKeyId: "##############"

    secretAccessKey: "##########"

 

    #

    # Whether to publish access keys. Default is false.

    # Publishing access keys to clients is necessary for those clients to have access to S3.

    # These keys are published in the configuration files of all HDFS clients.

    #

 

    publishAccessKeys: false

 

    #

    # ID of the Amazon AWS region to use

    # See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html

    #

 

    region: us-east-1

 

    #

    # Region endpoint (if you are using one of the Gov. regions)

    #

 

    # regionEndpoint: ec2.us-gov-west-1.amazonaws.com

 

    #

    # Name of the public key registered in AWS

    # See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html

    #

 

    keyName: "SE Sandbox"

 

    #

    # ID of the VPC subnet

    # See: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html

    #

 

    subnetId: subnet-6de2c61a

 

    #

    # Comma separated list of security group IDs

    # See: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html

    #

 

    securityGroupsIds: sg-c673b6a1

 

    #

    # A prefix that Cloudera Director should use when naming the instances (this is not part of the hostname)

    #

 

    instanceNamePrefix: demok16

 

    #

    # Specify a size for the root volume (in GBs). Cloudera Director will automatically expand the

    # filesystem so that you can use all the available disk space for your application

    # See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage_expand_partition.html

    #

 

    rootVolumeSizeGB: 100 # defaults to 50 GB if not specified

 

    #

    # Specify the type of the EBS volume used for the root partition. Defaults to gp2

    # See: http://aws.amazon.com/ebs/details/

    #

 

    rootVolumeType: gp2 # OR standard (for EBS magnetic)

 

    #

    # Whether to associate a public IP address with instances or not. If this is false

    # we expect instances to be able to access the internet using a NAT instance

    #

    # Currently the only way to get optimal S3 data transfer performance is to assign

    # public IP addresses to your instances and not use NAT (public subnet type of setup)

    #

    # See: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html

    #

 

    associatePublicIpAddresses: true

 

}

 

#

# SSH credentials to use to connect to the instances

#

 

ssh {

    username: root # for RHEL image

    privateKey: /root/projects/demok16-build/SESandbox.pem # with an absolute path to .pem file

}

 

#

# A list of instance types to use for group of nodes or management services

#

 

instances {

    m1x {

        type: "m1.xlarge"

 

        #

        # Amazon Machine Image (AMI)

        #

        # See: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html

        # Compatibility matrix: https://aws.amazon.com/amazon-linux-ami/instance-type-matrix/

        #

        # Red Hat Enterprise Linux AMI IDs: http://aws.amazon.com/partners/redhat/

        # We support Red Hat Enterprise Linux 6.4 (64bit) 64bit PV or HVM

        #

 

        image: ami-089e8360

 

        #

        # Name of the IAM Role to use for this instance type

        # http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

        #

 

        # iamProfileName: kirk

 

        tags {

            owner: "kirk"

        }

 

        bootstrapScript: """#!/bin/sh

 

# This is an embedded bootstrap script that runs as root and can be used to customize

# the instances immediately after boot and before any other Cloudera Director action

 

# If the exit code is not zero Cloudera Director will automatically retry

 

echo 'Hello World!'

exit 0

 

"""

    }

}

 

#

# Configuration for Cloudera Manager. Cloudera Director can use an existing instance

# or bootstrap everything from scratch for a new cluster

#

 

cloudera-manager {

 

    instance: ${instances.m1x} {

        tags {

            application: "Cloudera Manager 5"

        }

    }

 

    #

    # Automatically activate 60-Day Cloudera Enterprise Trial

    #

 

    enableEnterpriseTrial: false

 

}

 

#

# Cluster description

#

 

cluster {

 

    # List the products and their versions that need to be installed.

    # These products must have a corresponding parcel in the parcelRepositories

    # configured above. The specified version will be used to find a suitable

    # parcel. Specifying a version that points to more than one parcel among

    # those available will result in a configuration error. Specify more granular

    # versions to avoid conflicts.

 

    products {

      CDH: 5 # includes Impala and Spark

    }

 

    #

    # OR for CDH4

    #

    # products {

    #   CDH: 4

    #   IMPALA: 1.3

    #   SPARK: 0.9

    # }

 

    services: [HDFS, YARN, ZOOKEEPER, HBASE, HIVE]

 

    nodes {

        count: 5

        instance: ${instances.m1x}

    }

}

 

 

 

 

#################

 

 

Here's the data from the log file:

 

 

[2015-07-10 17:45:57] INFO  [main] - o.s.b.l.ClasspathLoggingApplicationListener: Application failed to start with classpath: [jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-boot-starter-1.1.8.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-boot-1.1.8.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-boot-autoconfigure-1.1.8.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-boot-starter-logging-1.1.8.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jcl-over-slf4j-1.7.7.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jul-to-slf4j-1.7.7.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/log4j-over-slf4j-1.7.7.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-core-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/snakeyaml-1.13.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-boot-starter-security-1.1.8.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-beans-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-context-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-expression-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-web-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-security-config-3.2.5.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/aopalliance-1.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-security-web-3.2.5.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-aop-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/javax.servlet-api-3.0.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/config-1.2.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/java-client-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-api-common-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/swagger-annotations-1.3.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-security-core-3.2.5.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-database-model-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/flyway-core-3.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-jobs-model-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-bootstrap-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-cloudera-manager-api-ext-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/cxf-rt-frontend-jaxrs-2.7.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/cxf-api-2.7.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/woodstox-core-asl-4.2.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/stax2-api-3.1.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/xmlschema-core-2.0.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/geronimo-javamail_1.4_spec-1.7.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/wsdl4j-1.6.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/cxf-rt-core-2.7.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jaxb-impl-2.1.13.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/javax.ws.rs-api-2.0-m10.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/cxf-rt-bindings-xml-2.7.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/cxf-rt-transports-http-2.7.5.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-pipeline-common-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-inspector-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/httpclient-4.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-collections4-4.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-validator-1.4.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-beanutils-1.9.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-collections-3.2.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-digester-2.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-common-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/guava-15.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-io-2.4.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/guava-retrying-1.0.6.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jsr305-2.0.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jackson-core-2.3.4.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jackson-databind-2.3.4.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jackson-annotations-2.3.4.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jackson-datatype-guava-2.2.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jackson-datatype-joda-2.3.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jbcrypt-0.3m.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/joda-time-2.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/metrics-core-3.1.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/aspectjweaver-1.8.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/objenesis-1.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-cleanup-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-health-cache-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/httpcore-4.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-api-v2-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-api-v1-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-aws-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-model-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-configs-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/com.amazonaws.aws-java-sdk-1.8.11.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-codec-1.4.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-sandbox-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-byon-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-pipeline-database-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-pipeline-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-orm-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-jdbc-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-tx-4.0.7.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-data-jpa-1.6.4.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/spring-data-commons-1.8.4.RELEASE.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/hibernate-entitymanager-4.3.6.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jboss-logging-3.1.3.GA.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jboss-logging-annotations-1.2.0.Beta1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/hibernate-core-4.3.6.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/antlr-2.7.7.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jandex-1.1.0.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/dom4j-1.6.1.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/xml-apis-1.0.b2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/hibernate-commons-annotations-4.0.5.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/hibernate-jpa-2.1-api-1.0.0.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/jboss-transaction-api_1.2_spec-1.0.0.Final.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/javassist-3.18.1-GA.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/h2-1.3.176.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-sshj-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/sshj-0.9.0.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/bcpkix-jdk15on-1.49.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/bcprov-jdk15on-1.49.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/launchpad-byodb-cm-1.1.3.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/logback-classic-1.1.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/logback-core-1.1.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/commons-lang3-3.3.2.jar!/, jar:file:/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar!/lib/slf4j-api-1.7.7.jar!/]

[2015-07-10 17:45:57] ERROR [main] - o.s.boot.SpringApplication: Application startup failed

com.typesafe.config.ConfigException$IO: ./demok16.aws.simple.conf: java.io.FileNotFoundException: ./demok16.aws.simple.conf (No such file or directory)

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:183) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:170) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parse(Parseable.java:148) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.SimpleIncluder.fromBasename(SimpleIncluder.java:172) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.ConfigImpl.parseFileAnySyntax(ConfigImpl.java:134) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.ConfigFactory.parseFileAnySyntax(ConfigFactory.java:606) ~[config-1.2.0.jar!/:na]

at com.cloudera.launchpad.ConfigurationInitializer.loadConfigWithConditionalDefaults(ConfigurationInitializer.java:68) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at com.cloudera.launchpad.ConfigurationInitializer.initialize(ConfigurationInitializer.java:54) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:572) ~[spring-boot-1.1.8.RELEASE.jar!/:1.1.8.RELEASE]

at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.1.8.RELEASE.jar!/:1.1.8.RELEASE]

at com.cloudera.launchpad.Application.start(Application.java:83) [launchpad-cli-1.1.3.jar!/:1.1.3]

at com.cloudera.launchpad.Application.main(Application.java:46) [launchpad-cli-1.1.3.jar!/:1.1.3]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_40]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_40]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_40]

at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_40]

at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [launchpad-cli-1.1.3.jar!/:1.1.3]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]

Caused by: java.io.FileNotFoundException: ./demok16.aws.simple.conf (No such file or directory)

at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_40]

at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_40]

at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[na:1.8.0_40]

at com.typesafe.config.impl.Parseable$ParseableFile.reader(Parseable.java:512) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:193) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:176) ~[config-1.2.0.jar!/:na]

... 17 common frames omitted

[2015-07-10 17:45:57] ERROR [main] - com.cloudera.launchpad.Application: Failed to initialize application context

com.typesafe.config.ConfigException$IO: ./demok16.aws.simple.conf: java.io.FileNotFoundException: ./demok16.aws.simple.conf (No such file or directory)

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:183) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:170) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parse(Parseable.java:148) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.SimpleIncluder.fromBasename(SimpleIncluder.java:172) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.ConfigImpl.parseFileAnySyntax(ConfigImpl.java:134) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.ConfigFactory.parseFileAnySyntax(ConfigFactory.java:606) ~[config-1.2.0.jar!/:na]

at com.cloudera.launchpad.ConfigurationInitializer.loadConfigWithConditionalDefaults(ConfigurationInitializer.java:68) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at com.cloudera.launchpad.ConfigurationInitializer.initialize(ConfigurationInitializer.java:54) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:572) ~[spring-boot-1.1.8.RELEASE.jar!/:1.1.8.RELEASE]

at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.1.8.RELEASE.jar!/:1.1.8.RELEASE]

at com.cloudera.launchpad.Application.start(Application.java:83) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at com.cloudera.launchpad.Application.main(Application.java:46) ~[launchpad-cli-1.1.3.jar!/:1.1.3]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_40]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_40]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_40]

at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_40]

at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [launchpad-cli-1.1.3.jar!/:1.1.3]

at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40]

Caused by: java.io.FileNotFoundException: ./demok16.aws.simple.conf (No such file or directory)

at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_40]

at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_40]

at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[na:1.8.0_40]

at com.typesafe.config.impl.Parseable$ParseableFile.reader(Parseable.java:512) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.rawParseValue(Parseable.java:193) ~[config-1.2.0.jar!/:na]

at com.typesafe.config.impl.Parseable.parseValue(Parseable.java:176) ~[config-1.2.0.jar!/:na]

... 17 common frames omitted

[2015-07-10 17:46:22] INFO  [main] - com.cloudera.launchpad.Application: Starting Application v1.1.3 on ip-10-0-0-173 with PID 10019 (/usr/lib64/cloudera-director/client/lib/launchpad-cli-1.1.3.jar started by root in /root/projects/demok16-build)

[2015-07-10 17:46:22] INFO  [main] - o.s.c.a.AnnotationConfigApplicationContext: Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@5192980b: startup date [Fri Jul 10 17:46:22 UTC 2015]; root of context hierarchy

[2015-07-10 17:46:27] INFO  [main] - c.c.launchpad.config.DatabaseConfig: Storing state on disk at /root/projects/demok16-build/./.demok16.aws.simple.conf

[2015-07-10 17:46:27] INFO  [main] - o.f.c.i.dbsupport.DbSupportFactory: Database: jdbc:h2:file:/root/projects/demok16-build/./.demok16.aws.simple.conf (H2 1.3)

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbValidate: Validated 2 migrations (execution time 00:00.024s)

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbMigrate: Current version of schema "PUBLIC": 3.1.1.0.1

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbMigrate: Schema "PUBLIC" is up to date. No migration necessary.

[2015-07-10 17:46:28] INFO  [main] - o.f.c.i.dbsupport.DbSupportFactory: Database: jdbc:h2:file:/root/projects/demok16-build/./.demok16.aws.simple.conf (H2 1.3)

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbValidate: Validated 7 migrations (execution time 00:00.024s)

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbMigrate: Current version of schema "PUBLIC": 3.1.1.2.1

[2015-07-10 17:46:28] INFO  [main] - o.f.core.internal.command.DbMigrate: Schema "PUBLIC" is up to date. No migration necessary.

[2015-07-10 17:46:28] INFO  [main] - o.s.o.j.LocalContainerEntityManagerFactoryBean: Building JPA container EntityManagerFactory for persistence unit 'default'

[2015-07-10 17:46:28] INFO  [main] - o.h.jpa.internal.util.LogHelper: HHH000204: Processing PersistenceUnitInfo [

name: default

...]

[2015-07-10 17:46:28] INFO  [main] - org.hibernate.Version: HHH000412: Hibernate Core {4.3.6.Final}

[2015-07-10 17:46:28] INFO  [main] - org.hibernate.cfg.Environment: HHH000206: hibernate.properties not found

[2015-07-10 17:46:28] INFO  [main] - org.hibernate.cfg.Environment: HHH000021: Bytecode provider name : javassist

[2015-07-10 17:46:29] INFO  [main] - o.h.annotations.common.Version: HCANN000001: Hibernate Commons Annotations {4.0.5.Final}

[2015-07-10 17:46:29] INFO  [main] - org.hibernate.dialect.Dialect: HHH000400: Using dialect: org.hibernate.dialect.H2Dialect

[2015-07-10 17:46:30] INFO  [main] - o.h.h.i.a.ASTQueryTranslatorFactory: HHH000397: Using ASTQueryTranslatorFactory

[2015-07-10 17:46:30] INFO  [main] - c.c.launchpad.config.MetricsConfig: Metrics reporting is disabled.

[2015-07-10 17:46:33] INFO  [main] - c.c.l.config.PipelineServiceConfig: Pipeline retry behavior: maxNumberOfAttempts=-1, maxWaitBetweenAttempts=45

[2015-07-10 17:46:35] INFO  [Thread-2] - o.s.c.a.AnnotationConfigApplicationContext: Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@5192980b: startup date [Fri Jul 10 17:46:22 UTC 2015]; root of context hierarchy

[2015-07-10 17:46:36] INFO  [Thread-2] - o.s.o.j.LocalContainerEntityManagerFactoryBean: Closing JPA EntityManagerFactory for persistence unit 'default'

[root@ip-10-0-0-173 logs]#

 

 

 

 

 

 

 

 

 

 

Thanks again - Kirk

avatar
Super Collaborator

The log you sent along seems to be for a different, failed run of the Director client where it couldn't find the configuration file you specified ("./demok16.aws.simple.conf"). There should have been a log generated for the successful run that bootstrapped the cluster you were talking about. It's possible that it was overwritten by a later run, or maybe you installed the client locally via tarball (if so, it's in a logs directory under where you decompressed it)?

 

The instance type m1.xlarge does come with 4 instance store (ephemeral) volumes of 420 GB each, so what you're seeing is normal for that instance type. Director preserves those volumes and doesn't currently provide a way to resize them. (Also, Director should be naming the devices /dev/sdb through /dev/sde, unless you've configured custom configuration values for lp.ec2.ephemeral.deviceNamePrefix and lp.ec2.ephemeral.rangeStart - the log would show some more information about that.)

 

If you don't need as much ephemeral storage, then you can try a different instance type - m1.xlarge is an older type and not recommended by AWS anyway.

 

  • m3.xlarge has the same vCPU count and memory as m1.xlarge (4, 15GB) but only 2 40GB SSD instance stores. It's also cheaper than m1.xlarge, according to AWS docs.
  • The new m4.xlarge has 4 vCPU, 16 GB memory, and does away with instance stores completely.