Cloudera Labs
Provide feedback on Cloudera Labs

Kafka errors installing via parcels

This is after I had to manually create the /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs directory on the nodes because the installed was getting 'permission denied'. Now, the kafka service fails to start with this error in /var/log/kafka/server.log:

 

2014-10-29 14:45:16,398 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)

 

And looking at the kafka.properties, it's right:

 

auto.create.topics.enable=true
controlled.shutdown.enable=true
default.replication.factor=3
kafka.log4j.dir=/var/log/kafka
log.dirs=/var/lib/kafka
log.retention.bytes=-1
log.retention.hours=168
log.roll.hours=168
log.segment.bytes=106954752
message.max.bytes=1000000
num.partitions=1
port=9092
replica.fetch.max.bytes=1048576
replica.lag.max.messages=4000
zookeeper.chroot=/kafka
zookeeper.session.timeout.ms=6000
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

 

(note the zookeeper.connect line is duplicated due to another bug - this line:

 

echo "zookeeper.connect=$QUORUM" >> $CONF_DIR/kafka.properties

 

in /var/run/cloudera-scm-agent/process/314-clabs_kafka-KAFKA_BROKER/scripts/control.sh means that zookeeper.connect will be appended every time the script is run, which is done multiple times in the case of it failing to start the first time)

 

 

If I manually set the broker.id and remove the duplicate zookeeper.connect lines in kafka.properties, and create the logs directory, kafka will start. However, the 'Add Service' workflow never completes, making it so I can't actually use it the "right way". Any ideas? Thanks!

17 REPLIES 17

Checking out the stderr log on one of the nodes gives me:

Thu Oct 30 15:55:21 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:23 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:26 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:30 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied

 

 

And looking at the stdout logs on the same server:

 

Thu Oct 30 15:55:21 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:21 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:23 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:23 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:26 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:26 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:30 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:30 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

I seem unable to paste in the stderr log. It seems to work, but then it doesn't show up when I refresh the message thread. So, I put it into a gist: https://gist.github.com/teancom/f83f536595bc3a7ef2f8

 

Moving on the, stdout log on the same server:

Thu Oct 30 15:55:21 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
Date: Thu Oct 30 15:55:21 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka
Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:23 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
Date: Thu Oct 30 15:55:23 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka
Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:26 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
Date: Thu Oct 30 15:55:26 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka
Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:30 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
Date: Thu Oct 30 15:55:30 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka
Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

I seem unable to paste in the stderr or stdout logs. It seems to work, but then it doesn't show up when I refresh the message thread. So, I put them into a gist: https://gist.github.com/teancom/f83f536595bc3a7ef2f8

 

Contributor

I think I managed to reproduce the issue and have a solution. 

It will sound wierd, but give it a try:

 

On every broker node that fails, run:

rm -r /var/lib/kafka/.mozilla

 

(actually, if there are any other directories in /var/lib/kafka, hidden or otherwise, nuke them too)

 

and then try to start Kafka from CM again.

 

Let me know how it goes!

There was no .mozilla directory on any of the nodes, but I deleted /var/lib/kafka and recreated, owned by the kafka user and group, and retried, to the same effect. 😕

Contributor
If the broker ID is still missing, you can add it through the CM UI.

Let me know if it works after that, or if there are still errors.

View solution in original post

I didn't know you could do that! I just added broker.ids to each node individually (I chose the last octet of their IP) and then started the service. And it worked! Yay! Thank you very much!

Contributor

I'm glad its working now!

 

Sorry it was such a PITA. We recorded the issues you reported and we'll improve in the next iteration of CLAB-KAFKA