Member since
05-24-2016
2
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3128 | 05-25-2016 11:28 PM |
05-25-2016
11:28 PM
I figured out the problem.
The default configurations use 'bootstrap.servers=sandbox.hortonworks.com:6667' instead of 'bootstrap.servers=localhost:9092' as the URL usage is required.
... View more
05-24-2016
11:19 PM
I am trying to use the Kafka Connect examples of write out to a file or console using the configuration files from within kafka's config folder [connect-console-sink.properties, config/connect-file-sink.properties] I have messages in the topics, made these changes in the config files too. Here's the last few entries from the log: [2016-05-24 23:18:15,241] INFO Finished creating connector local-console-source (org.apache.kafka.connect.runtime.Worker:193) [2016-05-24 23:18:15,245] INFO TaskConfig values: task.class = class org.apache.kafka.connect.file.FileStreamSinkTask (org.apache.kafka.connect.runtime.TaskConfig:165) [2016-05-24 23:18:15,245] INFO Creating task local-console-source-0 (org.apache.kafka.connect.runtime.Worker:256) [2016-05-24 23:18:15,246] INFO Instantiated task local-console-source-0 with version 0.9.0.2.4.0.0-169 of type org.apache.kafka.connect.file.FileStreamSinkTask (org.apache.kafka.connect.runtime.Worker:267) [2016-05-24 23:18:15,253] INFO ConsumerConfig values: request.timeout.ms = 40000 check.crcs = true retry.backoff.ms = 100 ssl.truststore.password = null ssl.keymanager.algorithm = SunX509 receive.buffer.bytes = 32768 ssl.cipher.suites = null ssl.key.password = null sasl.kerberos.ticket.renew.jitter = 0.05 ssl.provider = null sasl.kerberos.service.name = null session.timeout.ms = 30000 sasl.kerberos.ticket.renew.window.factor = 0.8 bootstrap.servers = [localhost:9092] client.id = fetch.max.wait.ms = 500 fetch.min.bytes = 1024 key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer sasl.kerberos.kinit.cmd = /usr/bin/kinit auto.offset.reset = earliest value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] partition.assignment.strategy = [org.apache.kafka.clients.consumer.RangeAssignor] ssl.endpoint.identification.algorithm = null max.partition.fetch.bytes = 1048576 ssl.keystore.location = null ssl.truststore.location = null ssl.keystore.password = null metrics.sample.window.ms = 30000 metadata.max.age.ms = 300000 security.protocol = PLAINTEXT auto.commit.interval.ms = 5000 ssl.protocol = TLS sasl.kerberos.min.time.before.relogin = 60000 connections.max.idle.ms = 540000 ssl.trustmanager.algorithm = PKIX group.id = connect-local-console-source enable.auto.commit = false metric.reporters = [] ssl.truststore.type = JKS send.buffer.bytes = 131072 reconnect.backoff.ms = 50 metrics.num.samples = 2 ssl.keystore.type = JKS heartbeat.interval.ms = 3000 (org.apache.kafka.clients.consumer.ConsumerConfig:165) [2016-05-24 23:18:15,286] INFO Kafka version : 0.9.0.2.4.0.0-169 (org.apache.kafka.common.utils.AppInfoParser:82) [2016-05-24 23:18:15,286] INFO Kafka commitId : 29fa247911f6823b (org.apache.kafka.common.utils.AppInfoParser:83) [2016-05-24 23:18:15,290] INFO Created connector local-console-source (org.apache.kafka.connect.cli.ConnectStandalone:82) ^C [2016-05-24 23:19:01,375] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:68) [2016-05-24 23:19:01,375] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:136) [2016-05-24 23:19:01,381] INFO Stopped ServerConnector@55ecdfba{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:306) [2016-05-24 23:19:01,388] INFO Stopped o.e.j.s.ServletContextHandler@3b907954{/,null,UNAVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:865) [2016-05-24 23:19:01,390] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:147) [2016-05-24 23:19:01,390] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:62) [2016-05-24 23:19:01,390] INFO Stopping task local-console-source-0 (org.apache.kafka.connect.runtime.Worker:305) [2016-05-24 23:19:01,391] INFO Starting graceful shutdown of thread WorkerSinkTask-local-console-source-0 (org.apache.kafka.connect.util.ShutdownableThread:119) [2016-05-24 23:19:01,391] ERROR Sink task WorkerSinkTask{id=local-console-source-0} was stopped before completing join group. Task initialization and start is being skipped (org.apache.kafka.connect.runtime.WorkerSinkTask:150) Exception in thread "Thread-1" java.lang.NullPointerException at org.apache.kafka.connect.file.FileStreamSinkTask.stop(FileStreamSinkTask.java:88) at org.apache.kafka.connect.runtime.WorkerSinkTask.awaitStop(WorkerSinkTask.java:119) at org.apache.kafka.connect.runtime.Worker.stopTask(Worker.java:311) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.removeConnectorTasks(StandaloneHerder.java:238) at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.stop(StandaloneHerder.java:68) at org.apache.kafka.connect.runtime.Connect.stop(Connect.java:71) at org.apache.kafka.connect.runtime.Connect$ShutdownHook.run(Connect.java:93) [root@sandbox bin]#
... View more
Labels:
- Labels:
-
Apache Kafka