Member since
05-21-2018
14
Posts
0
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2844 | 06-21-2018 01:10 PM | |
| 1747 | 06-18-2018 09:19 AM |
01-30-2019
07:51 AM
I am using CentOS 7 on AWS. The image is ami-e1496384 in the us-east-2 region. I will try setting unlimited-jce to false.
... View more
01-30-2019
07:34 AM
I am trying to bootstrap a CDH6.1 cluster using Director 6.1. I have set Director's application.properties like so:
lp.bootstrap.packages.defaultCmJavaPackage:java-1.8.0-openjdk-devel
lp.bootstrap.packages.cmJavaPackages[0]: "6\\..*=java-1.8.0-openjdk-devel"
lp.bootstrap.packages.cmJavaPackages[1]: "5\\..*=java-1.7.0-openjdk-devel"
The cluster fails with the following error in Director's application.log:
[2019-01-30 14:21:34.311 +0000] INFO [io-thread-7] - - - - - ssh:10.41.1.10: /usr/lib/jvm/java-openjdk
[2019-01-30 14:21:34.311 +0000] INFO [p-2814a062ee12-DefaultBootstrapDeploymentJob] 2125503a-9188-4da6-b8df-22f83aebe3fe POST /api/d6.1/environments/HDFS-Kudu-Cluster-DEV-2%20Environment/deployments com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce - c.c.l.inspector.util.JdkInspector: Found JAVA_HOME: /usr/lib/jvm/java-openjdk
[2019-01-30 14:21:34.453 +0000] INFO [io-thread-6] - - - - - ssh:10.41.1.10: openjdk version "1.8.0_191"
[2019-01-30 14:21:34.453 +0000] INFO [io-thread-6] - - - - - ssh:10.41.1.10: OpenJDK Runtime Environment (build 1.8.0_191-b12)
[2019-01-30 14:21:34.453 +0000] INFO [io-thread-6] - - - - - ssh:10.41.1.10: OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
[2019-01-30 14:21:34.585 +0000] INFO [io-thread-6] - - - - - ssh:10.41.1.10: javac 1.8.0_191
[2019-01-30 14:21:34.643 +0000] INFO [p-2814a062ee12-DefaultBootstrapDeploymentJob] 2125503a-9188-4da6-b8df-22f83aebe3fe POST /api/d6.1/environments/HDFS-Kudu-Cluster-DEV-2%20Environment/deployments com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce - c.c.l.b.EnableUnlimitedStrengthJce: Detected Java version: null
[2019-01-30 14:21:34.716 +0000] INFO [p-2814a062ee12-DefaultBootstrapDeploymentJob] 2125503a-9188-4da6-b8df-22f83aebe3fe POST /api/d6.1/environments/HDFS-Kudu-Cluster-DEV-2%20Environment/deployments com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce - c.c.l.b.EnableUnlimitedStrengthJce: Version null of Java is not supported for unlimited strength policy files
[2019-01-30 14:21:34.722 +0000] ERROR [p-2814a062ee12-DefaultBootstrapDeploymentJob] 2125503a-9188-4da6-b8df-22f83aebe3fe POST /api/d6.1/environments/HDFS-Kudu-Cluster-DEV-2%20Environment/deployments com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: null
at com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce.run(EnableUnlimitedStrengthJce.java:153)
at com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce.run(EnableUnlimitedStrengthJce.java:48)
at com.cloudera.launchpad.pipeline.job.Job2.runUnchecked(Job2.java:31)
at com.cloudera.launchpad.pipeline.job.Job2$$FastClassBySpringCGLIB$$54178502.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:746)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at com.cloudera.launchpad.pipeline.PipelineJobProfiler.profileJobRun(PipelineJobProfiler.java:60)
at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
at com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce$$EnhancerBySpringCGLIB$$1504d726.runUnchecked(<generated>)
at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:202)
at com.cloudera.launchpad.pipeline.util.PipelineRunner$JobCallable.call(PipelineRunner.java:173)
at com.github.rholder.retry.AttemptTimeLimiters$NoAttemptTimeLimit.call(AttemptTimeLimiters.java:78)
at com.github.rholder.retry.Retryer.call(Retryer.java:160)
at com.cloudera.launchpad.pipeline.util.PipelineRunner.attemptMultipleJobExecutionsWithRetries(PipelineRunner.java:136)
at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.doRun(DatabasePipelineRunner.java:201)
at com.cloudera.launchpad.pipeline.DatabasePipelineRunner.run(DatabasePipelineRunner.java:141)
at com.cloudera.launchpad.ExceptionHandlingRunnable.run(ExceptionHandlingRunnable.java:57)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[2019-01-30 14:21:34.727 +0000] ERROR [p-2814a062ee12-DefaultBootstrapDeploymentJob] 2125503a-9188-4da6-b8df-22f83aebe3fe POST /api/d6.1/environments/HDFS-Kudu-Cluster-DEV-2%20Environment/deployments com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error: JobError{jobClassName=com.cloudera.launchpad.bootstrap.EnableUnlimitedStrengthJce, jobContext=JobContext{callCountAtThisStackLevel=0, pipelineHandle='553dab04-9a81-4ae3-af11-2814a062ee12', callStack=CallStack{items=[Item{className='com.cloudera.launchpad.api.jobs.DefaultBootstrapDeploymentJob', callCount=7}, Item{className='com.cloudera.launchpad.bootstrap.deployment.BootstrapClouderaManager', callCount=5}], size=2, parent=Optional.absent()}, stackLevel=2}, errorInfo=ErrorInfo{code=POLICY_FILE_INSTALLATION_FAIL, properties={instance=e5755692-0d3a-4769-add0-6a28cf355314, javaHome=/usr/lib/jvm/java-openjdk, javaMajorVersion=unsupported}, causes=[]}}
com.cloudera.launchpad.pipeline.UnrecoverablePipelineError: null
... View more
Labels:
- Labels:
-
Apache Kudu
-
HDFS
10-01-2018
07:40 AM
You should be able to access it via HTTP on port 7180 (using basic auth). I'm not sure if HTTPS is available at all. Is 7183 supposed to be for HTTPS? You will find API help in the Manager... upper right> Support > API Documentation / API Explorer. This is on 6.0. Not sure about other version.
... View more
09-21-2018
06:27 AM
I should mention I tried flipping this flag to false but got the same results. This is the only thing that looked like it might be necessary. exclude.internal.topics = false
... View more
09-21-2018
06:16 AM
I am using the newly release Cloudera 6. I have a Kafka node with Zookeeper setup. I am able to produce messages, but unable to consume messages. Here is how I am producing messages: $ kafka-console-producer --batch-size 1 --broker-list <hostname>:9092 --topic TEST
... ProducerConfig values:
acks = 1
batch.size = 16384
bootstrap.servers = [<hostname>:9092]
buffer.memory = 33554432
client.id = console-producer
compression.type = none
connections.max.idle.ms = 540000
enable.idempotence = false
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 1000
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 1500
retries = 3
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 102400
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
transaction.timeout.ms = 60000
transactional.id = null
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
18/09/20 19:18:56 INFO utils.AppInfoParser: Kafka version : 1.0.1-cdh6.0.0
18/09/20 19:18:56 INFO utils.AppInfoParser: Kafka commitId : unknown
>TEST MESSAGE
>TEST MESSAGE 2
> Trying to consum messages: $ kafka-console-consumer --bootstrap-server <hostname>:9092 --topic TEST --from-beginning
....
18/09/20 19:18:07 INFO consumer.ConsumerConfig: ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [hostname:9092]
check.crcs = true
client.id =
connections.max.idle.ms = 540000
enable.auto.commit = true
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = console-consumer-77498
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = true
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
18/09/20 19:18:07 INFO utils.AppInfoParser: Kafka version : 1.0.1-cdh6.0.0
18/09/20 19:18:07 INFO utils.AppInfoParser: Kafka commitId : unknown No messages appear to be consumed (expectation is that they would print out above). We have the same behavior in our Java Spring application.
... View more
Labels:
- Labels:
-
Apache Kafka
-
Apache Zookeeper
09-20-2018
11:58 AM
Are you able to send messages from the command line tool like so: $ kafka-console-producer --broker-list <yourhost>:9092 --topic test-topic
... View more
07-16-2018
08:24 AM
Does Cloudera Director actually have the ability to install Kudu, either via GUI or API? I thought I installed everything but not Kudu. The link specified is for Manager
... View more
06-21-2018
01:10 PM
We figured this one out. The nodes were not able to download the parcels from the manager instance. As it turns out we had DNS Hostnames set to NO for the VPC (uAWS). The message that tipped us off was in the cloudera-scm-agent.log on each of the nodes. [21/Jun/2018 19:42:05 +0000] 13795 Thread-13 downloader INFO Fetching torrent: http://ip-10-2-4-152.us-east-2.compute.internal:7180/cmf/parcel/download/KAFKA-3.0.0-1.3.0.0.p0.40-el7.parcel.torrent
[21/Jun/2018 19:42:05 +0000] 13795 Thread-13 https ERROR Failed to retrieve/stroe URL: http://ip-10-2-4-152.us-east-2.compute.internal:7180/cmf/parcel/download/KAFKA-3.0.0-1.3.0.0.p0.40-el7.parcel.torrent -> /opt/cloudera/parcel-cache/KAFKA-3.0.0-1.3.0.0.p0.40-el7.parcel.torrent <urlopen error [Errno -2] Name or service not known>
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.15.0-py2.7.egg/cmf/https.py", line 191, in fetch_to_file
resp = self.open(req_url)
File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.15.0-py2.7.egg/cmf/https.py", line 186, in open
... View more
06-21-2018
07:54 AM
I am trying to deploy a cluster in AWS using Cloudera Director. It appears to go smooth right up until the end with 579 / 597 steps completed. GUI indicates: "Distributing parcels: KAFKA-3.0.0-1.3.0.0.p0.40,CDH-5.15.0-1.cdh5.15.0.p0.21" cloudera-director-server indicates over and over : 51.538 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:50:53.552 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:50:55.565 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:50:57.579 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:50:59.595 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:01.609 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:03.624 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:05.675 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:07.689 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:09.704 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:11.718 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:13.732 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null}
[2018-06-21 14:51:15.745 +0000] INFO [p-c86a21fc7a0e-DefaultBootstrapClusterJob] 1b43a3bf-7aa4-4d09-bf79-ed5f30552520 POST /api/v12/import com.cloudera.launchpad.bootstrap.cluster.UnboundedWaitForParcelStage - c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (KAFKA, 3.0.0-1.3.0.0.p0.40) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=1600, count=0, countTotal=4, warnings=null, errors=null} Edit: this doesn't appear to be KAFKA specific issue. I tried another cluster w/o KAFKA and this time it hangs on a CDH parcel: c.c.l.b.c.UnboundedWaitForParcelStage: Waiting for parcel (CDH, 5.15.0-1.cdh5.15.0.p0.21) stage DISTRIBUTED. Current: DISTRIBUTING Past: [DISTRIBUTING, DISTRIBUTING, DISTRIBUTING]. State ApiParcelState{progress=0, progressTotal=4900, count=0, countTotal=7, warnings=null, errors=null}
... View more
Labels:
- Labels:
-
Apache Kafka
06-18-2018
09:19 AM
OK I figured this missing piece (at least for AWS) to be: {
'accessKeyId': "accessKeyId",
'secretAccessKey': "secretAccessKey",
'region': "region"
} I figured this out using the Cloudera's python example here: https://github.com/cloudera/director-sdk/blob/master/python-client-samples/cluster.py
... View more
06-18-2018
07:35 AM
I am trying to use the Create Environment API for cluster automation, which I believe to be the first step (based on the GUI) in setting up my cluster. {
"name": "",
"provider": {
"type": "",
"config": "object"
},
"credentials": {
"username": "",
"password": "",
"privateKey": "",
"passphrase": "",
"port": 0,
"hostKeyFingerprint": "",
"bastionHost": ""
}
} First off, am I on the right track? And if so where do I find details on this portion which appears to be the section for my clouder provider details? config (object): Provider specific configurations [redacted on read] I am using AWS.
... View more
