Reply
Explorer
Posts: 16
Registered: ‎10-29-2014
Accepted Solution

Kafka errors installing via parcels

This is after I had to manually create the /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs directory on the nodes because the installed was getting 'permission denied'. Now, the kafka service fails to start with this error in /var/log/kafka/server.log:

 

2014-10-29 14:45:16,398 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)

 

And looking at the kafka.properties, it's right:

 

auto.create.topics.enable=true
controlled.shutdown.enable=true
default.replication.factor=3
kafka.log4j.dir=/var/log/kafka
log.dirs=/var/lib/kafka
log.retention.bytes=-1
log.retention.hours=168
log.roll.hours=168
log.segment.bytes=106954752
message.max.bytes=1000000
num.partitions=1
port=9092
replica.fetch.max.bytes=1048576
replica.lag.max.messages=4000
zookeeper.chroot=/kafka
zookeeper.session.timeout.ms=6000
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

 

(note the zookeeper.connect line is duplicated due to another bug - this line:

 

echo "zookeeper.connect=$QUORUM" >> $CONF_DIR/kafka.properties

 

in /var/run/cloudera-scm-agent/process/314-clabs_kafka-KAFKA_BROKER/scripts/control.sh means that zookeeper.connect will be appended every time the script is run, which is done multiple times in the case of it failing to start the first time)

 

 

If I manually set the broker.id and remove the duplicate zookeeper.connect lines in kafka.properties, and create the logs directory, kafka will start. However, the 'Add Service' workflow never completes, making it so I can't actually use it the "right way". Any ideas? Thanks!

Cloudera Employee
Posts: 31
Registered: ‎10-16-2013

Re: Kafka errors installing via parcels

Thanks for the very complete information!

From your description, it looks like the Parcel was installed, but adding the Kafka Service and its brokers through CM failed and you are trying to work around that, did I get it right?

 

Unfortunately, we didn't write our scripts to support this scenario.

 

Specifically, the broker ID is generated for each broker by the CM Kafka Service (so we can avoid duplicates), which is probably why you don't see it. Also, CM does not plan on running the same control.sh with same properties file multiple times, we generate fresh copies on restarts.

 

So, being unable to successfully run Kafka from command line after installing the Parcel is expected (but unfortunate. We'll work on improving that). 

 

We can focus on getting the service to deploy successfully. Can you check CM server and CM agent for errors? Perhaps there are other missing permissions? Perhaps removing the parcel, and re-deploying will be the best bet now, because we are not sure about the current state.

Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

Well, I wasn't trying necessarily to work around it not adding the service, I was trying to figure out why it was failing. If I remove everything, redownload the parcel, activate, restart the cluster, and then try to add kafka as a service, I run into this issue (again):

 

mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied

 

That's from the 'stderr' log on all three nodes that I designated. I also checked and the cloudera-scm-agent is running as root on the machines:

 

root 2345 0.2 0.1 2017764 50436 ? Ssl Oct28 3:57 /usr/lib64/cmf/agent/build/env/bin/python /usr/lib64/cmf/agent/src/cmf/agent.py --package_dir /usr/lib64/cmf/service --agent_dir /var/run/cloudera-scm-agent --lib_dir /var/lib/cloudera-scm-agent --logfile /var/log/cloudera-scm-agent/cloudera-scm-agent.log
root 2378 0.0 0.0 202196 12676 ? Ss Oct28 0:25 /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/python /usr/lib64/cmf/agent/src/cmf/../../build/env/bin/supervisord
root 2379 0.0 0.0 195964 12348 ? S Oct28 0:00 \_ /usr/lib64/cmf/agent/build/env/bin/python /usr/lib64/cmf/agent/src/cmf/supervisor_listener.py -l /var/log/cloudera-scm-agent/cmf_listener.log /var/run/cloudera-scm-agent/events

Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

Any idea where I should go from here? If I manually create the logs directory, I'm quite sure I will just run into the same problem that I did before.

Highlighted
Cloudera Employee
Posts: 20
Registered: ‎07-08-2013

Re: Kafka errors installing via parcels

[ Edited ]

Thank you for the detailed information David, really helpful!

I've looked at my testing cluster and I do see the same error line:

mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied

The issue is that the binary script is calling "mkdir" on directory that is not actually used and hence you can safely ignore this message.

What CDH and CM version are you using? Only CM 5.2 and up will work.

Could you share kafka logs from /var/log/kafka/ and all the configuration files from /var/run/cloudera-scm-agent/process/$fill-me-in directory?

Cloudera Employee
Posts: 20
Registered: ‎07-08-2013

Re: Kafka errors installing via parcels

FYI: I've filled an internal JIRA to remove the harmless error message.
Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

Well, don't I feel silly. I was following the instructions at http://blogs.aws.amazon.com/bigdata/post/Tx2D0J7QOVRJBRX/Deploying-Cloudera-s-Enterprise-Data-Hub-on... and forgot that I had to install 5.1.3 to work around the missing mysql jar bug. I apologize for wasting your time.

Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

Ugh, ugh, ugh. Now I really do feel like an idiot. I *am* running 5.2.0, I'm 100% certain. Apparently, I need *two* cups of coffee before using technology in the morning. I'll follow up with the logs requested in a little bit.

Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

Let's start with my verification that I'm running 5.2.0 -
http://cl.ly/image/0m0c0J3k3Y3H

Here are all the lines from /var/log/kafka/server.log:

2014-10-30 15:55:21,961 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:23,864 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:26,730 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:30,595 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)

Here is the contents of /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/kafka.properties:

auto.create.topics.enable=true
controlled.shutdown.enable=true
default.replication.factor=3
kafka.log4j.dir=/var/log/kafka
log.dirs=/var/lib/kafka
log.retention.bytes=-1
log.retention.hours=168
log.roll.hours=168
log.segment.bytes=106954752
message.max.bytes=1000000
num.partitions=1
port=9092
replica.fetch.max.bytes=1048576
replica.lag.max.messages=4000
zookeeper.chroot=/kafka
zookeeper.session.timeout.ms=6000
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

This is on a machine that has a completely fresh install of kakfa on it. Specifically, I'm on step 4 of the "add a service" wizard - http://cl.ly/image/1w371h1d2B3p

If I click 'details', it pops up another window that looks like - http://cl.ly/image/471E3Y3j2d3I

Checking out the stderr log on one of the nodes gives me:

Thu Oct 30 15:55:21 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:23 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:26 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied
Thu Oct 30 15:55:30 EDT 2014
+ locate_java_home
+ '[' -z '' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e '/usr/lib/jvm/java-7-oracle*/bin/java' ']'
+ for candidate in '/usr/lib/jvm/java-7-oracle*' '/usr/java/jdk1.7*' '/usr/java/jre1.7*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*' '/usr/lib/jvm/java-1.7.0-oracle-1.7.0.*/jre/' /usr/lib/jvm/j2sdk1.7-oracle /usr/lib/jvm/j2sdk1.7-oracle/jre /usr/lib/j2sdk1.6-sun /usr/lib/jvm/java-6-sun '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*' '/usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/' /usr/lib/jvm/j2sdk1.6-oracle /usr/lib/jvm/j2sdk1.6-oracle/jre '/usr/java/jdk1.6*' '/usr/java/jre1.6*' /Library/Java/Home /usr/java/default /usr/lib/jvm/default-java /usr/lib/jvm/java-openjdk /usr/lib/jvm/jre-openjdk /usr/lib/jvm/java '/usr/lib/jvm/java-1.7.0-openjdk*' '/usr/lib/jvm/java-7-openjdk*' /usr/lib/jvm/java-1.6.0-openjdk '/usr/lib/jvm/java-1.6.0-openjdk-*' '/usr/lib/jvm/jre-1.6.0-openjdk*'
+ '[' -e /usr/java/jdk1.7.0_55-cloudera/bin/java ']'
+ export JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ break
+ verify_java_home
+ '[' -z /usr/java/jdk1.7.0_55-cloudera ']'
+ echo JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh:/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=2
++ seq 1 2
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
+ PARCEL_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
+ . /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/meta/kafka_env.sh
++ KAFKA_DIRNAME=CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14
++ export CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
++ CLABS_KAFKA_HOME=/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
+ for i in '`seq 1 $COUNT`'
+ SCRIPT=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
+ . /opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-5.2.0-1.cdh5.2.0.p0.36
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hive-hcatalog
++ export CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ CDH_SQOOP2_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sqoop2
++ export CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ CDH_LLAMA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/llama
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/sentry
++ export TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ TOMCAT_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-tomcat
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-5.2.0-1.cdh5.2.0.p0.36/lib/hadoop-kms
+ echo 'Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir'
+ echo 'Using scripts/control.sh as process script'
+ chmod u+x /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh
+ exec /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/scripts/control.sh start
mkdir: cannot create directory `/opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka/bin/../logs': Permission denied

And looking at the stdout logs on the same server:

Thu Oct 30 15:55:21 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:21 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:23 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:23 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:26 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:26 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
Thu Oct 30 15:55:30 EDT 2014
JAVA_HOME=/usr/java/jdk1.7.0_55-cloudera
Using /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script

Date: Thu Oct 30 15:55:30 EDT 2014
Host: ip-10-10-9-144.us-west-2.compute.internal
Pwd: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CONF_DIR: /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER
CLABS_KAFKA_HOME: /opt/cloudera/parcels/CLABS_KAFKA-0.8.1.1-1.kafka1.0.0.p0.14/lib/kafka
Zoookeper Quorum: ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181
Chroot: /kafka

Final Zookeeper Quorum is ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

Explorer
Posts: 16
Registered: ‎10-29-2014

Re: Kafka errors installing via parcels

I seem to being having a problem putting everything into one comment, so I'm going to split it up a bit.

 

Let's start with my verification that I'm running 5.2.0 -
http://cl.ly/image/0m0c0J3k3Y3H

Here are all the lines from /var/log/kafka/server.log:

2014-10-30 15:55:21,961 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:23,864 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:26,730 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)
2014-10-30 15:55:30,595 FATAL kafka.Kafka$:
java.lang.IllegalArgumentException: requirement failed: Missing required property 'broker.id'
at scala.Predef$.require(Predef.scala:233)
at kafka.utils.VerifiableProperties.getIntInRange(VerifiableProperties.scala:45)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:49)
at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:31)
at kafka.Kafka$.main(Kafka.scala:35)
at kafka.Kafka.main(Kafka.scala)

 

Here are the contents of /var/run/cloudera-scm-agent/process/357-clabs_kafka-KAFKA_BROKER/kafka.properties:

auto.create.topics.enable=true
controlled.shutdown.enable=true
default.replication.factor=3
kafka.log4j.dir=/var/log/kafka
log.dirs=/var/lib/kafka
log.retention.bytes=-1
log.retention.hours=168
log.roll.hours=168
log.segment.bytes=106954752
message.max.bytes=1000000
num.partitions=1
port=9092
replica.fetch.max.bytes=1048576
replica.lag.max.messages=4000
zookeeper.chroot=/kafka
zookeeper.session.timeout.ms=6000
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka
zookeeper.connect=ip-10-10-9-139.us-west-2.compute.internal:2181,ip-10-10-9-140.us-west-2.compute.internal:2181,ip-10-10-9-141.us-west-2.compute.internal:2181/kafka

 

This is on a machine that has a completely fresh install of kakfa on it. Specifically, I'm on step 4 of the "add a service" wizard - http://cl.ly/image/1w371h1d2B3p

 

If I click 'details', it pops up another window that looks like - http://cl.ly/image/471E3Y3j2d3I

Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB