Member since
08-29-2022
2
Posts
0
Kudos Received
0
Solutions
01-07-2023
10:05 PM
Want to know your opinion about next situation:
After that I installed (Ambari 3.1 and Enabled Kerberos + edited Configuration for Kafka) everything and managed to initiate topic, added some data everything looks fine, but when I am trying to import some heavy data like big CSV into data getting Error, in Debug mode shown below:
kafka-console-producer.sh --broker-list test.kafka:6667 --topic cleanCsv --producer-property security.protocol=SASL_PLAINTEXT < /tmp/clean_csv_full.csv Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 [2023-01-08 03:23:06,861] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$) [2023-01-08 03:23:06,922] INFO ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [test.kafka:6667] buffer.memory = 33554432 client.id = console-producer compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = [] key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer linger.ms = 1000 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner receive.buffer.bytes = 32768 reconnect.backoff.max.ms = 1000 reconnect.backoff.ms = 50 request.timeout.ms = 1500 retries = 3 retry.backoff.ms = 100 sasl.client.callback.handler.class = null sasl.jaas.config = null sasl.kerberos.kinit.cmd = /usr/bin/kinit sasl.kerberos.min.time.before.relogin = 60000 sasl.kerberos.service.name = null sasl.kerberos.ticket.renew.jitter = 0.05 sasl.kerberos.ticket.renew.window.factor = 0.8 sasl.login.callback.handler.class = null sasl.login.class = null sasl.login.refresh.buffer.seconds = 300 sasl.login.refresh.min.period.seconds = 60 sasl.login.refresh.window.factor = 0.8 sasl.login.refresh.window.jitter = 0.05 sasl.mechanism = GSSAPI security.protocol = SASL_PLAINTEXT send.buffer.bytes = 102400 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = https ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer (org.apache.kafka.clients.producer.ProducerConfig) [2023-01-08 03:23:06,939] DEBUG Added sensor with name bufferpool-wait-time (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:06,942] DEBUG Added sensor with name buffer-exhausted-records (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:06,945] DEBUG Updated cluster metadata version 1 to Cluster(id = null, nodes = [test.kafka:6667 (id: -1 rack: null)], partitions = [], controller = null) (org.apache.kafka.clients.Metadata) [2023-01-08 03:23:07,011] INFO Successfully logged in. (org.apache.kafka.common.security.authenticator.AbstractLogin) [2023-01-08 03:23:07,012] DEBUG [Principal=null]: It is a Kerberos ticket (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,014] INFO [Principal=null]: TGT refresh thread started. (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,014] DEBUG Found TGT with client principal 'kafka/test.kafka@TEST' and server principal 'krbtgt/TEST@TEST'. (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,017] DEBUG Added sensor with name produce-throttle-time (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,024] INFO [Principal=null]: TGT valid starting at: Sun Jan 08 03:15:26 UTC 2023 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,025] INFO [Principal=null]: TGT expires: Sun Jan 08 13:15:26 UTC 2023 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,025] INFO [Principal=null]: TGT refresh sleeping until: Sun Jan 08 11:42:44 UTC 2023 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:07,026] DEBUG Added sensor with name connections-closed: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,027] DEBUG Added sensor with name connections-created: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,031] DEBUG Added sensor with name successful-authentication: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,031] DEBUG Added sensor with name failed-authentication: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,032] DEBUG Added sensor with name bytes-sent-received: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,032] DEBUG Added sensor with name bytes-sent: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,033] DEBUG Added sensor with name bytes-received: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,034] DEBUG Added sensor with name select-time: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,035] DEBUG Added sensor with name io-time: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,039] DEBUG Added sensor with name batch-size (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,039] DEBUG Added sensor with name compression-rate (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,039] DEBUG Added sensor with name queue-time (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,039] DEBUG Added sensor with name request-time (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,040] DEBUG Added sensor with name records-per-request (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,040] DEBUG Added sensor with name record-retries (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,040] DEBUG Added sensor with name errors (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,041] DEBUG Added sensor with name record-size (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,042] DEBUG Added sensor with name batch-split-rate (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,042] DEBUG [Producer clientId=console-producer] Starting Kafka producer I/O thread. (org.apache.kafka.clients.producer.internals.Sender) [2023-01-08 03:23:07,044] INFO Kafka version : 2.0.0.3.1.5.6091-7 (org.apache.kafka.common.utils.AppInfoParser) [2023-01-08 03:23:07,044] INFO Kafka commitId : a10a6e16779f1930 (org.apache.kafka.common.utils.AppInfoParser) [2023-01-08 03:23:07,045] DEBUG [Producer clientId=console-producer] Kafka producer started (org.apache.kafka.clients.producer.KafkaProducer) >[2023-01-08 03:23:07,062] DEBUG [Producer clientId=console-producer] Initialize connection to node test.kafka:6667 (id: -1 rack: null) for sending metadata request (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,062] DEBUG [Producer clientId=console-producer] Initiating connection to node test.kafka:6667 (id: -1 rack: null) (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,069] DEBUG Set SASL client state to SEND_APIVERSIONS_REQUEST (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,070] DEBUG Creating SaslClient: client=kafka/test.kafka@TEST;service=kafka;serviceHostname=test.kafka;mechs=[GSSAPI] (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,105] DEBUG Added sensor with name node--1.bytes-sent (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,106] DEBUG Added sensor with name node--1.bytes-received (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,106] DEBUG Added sensor with name node--1.latency (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:07,107] DEBUG [Producer clientId=console-producer] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 102400, SO_TIMEOUT = 0 to node -1 (org.apache.kafka.common.network.Selector) [2023-01-08 03:23:07,187] DEBUG Set SASL client state to RECEIVE_APIVERSIONS_RESPONSE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,188] DEBUG [Producer clientId=console-producer] Completed connection to node -1. Fetching API versions. (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,190] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,190] DEBUG Set SASL client state to RECEIVE_HANDSHAKE_RESPONSE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,191] DEBUG Set SASL client state to INITIAL (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,307] DEBUG Set SASL client state to INTERMEDIATE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,310] DEBUG Set SASL client state to CLIENT_COMPLETE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,310] DEBUG Set SASL client state to COMPLETE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,310] DEBUG [Producer clientId=console-producer] Initiating API versions fetch from node -1. (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,314] DEBUG [Producer clientId=console-producer] Recorded API versions for node -1: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8 [usable: 8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable: 6], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable: 1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4], FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3], Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2], SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2], ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1], ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3], DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1], InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1 [usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0 to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0 [usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1 [usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1 [usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1 [usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35): 0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37): 0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1], RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1 [usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42): 0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,314] DEBUG [Producer clientId=console-producer] Sending metadata request (type=MetadataRequest, topics=cleanCsv) to node test.kafka:6667 (id: -1 rack: null) (org.apache.kafka.clients.NetworkClient) [2023-01-08 03:23:07,321] INFO Cluster ID: g5oEj_9HRpS5nx3hsiK-hA (org.apache.kafka.clients.Metadata) [2023-01-08 03:23:07,321] DEBUG Updated cluster metadata version 2 to Cluster(id = g5oEj_9HRpS5nx3hsiK-hA, nodes = [test.kafka:6667 (id: 1001 rack: null)], partitions = [Partition(topic = cleanCsv, partition = 0, leader = 1001, replicas = [1001], isr = [1001], offlineReplicas = [])], controller = test.kafka:6667 (id: 1001 rack: null)) (org.apache.kafka.clients.Metadata) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>[2023-01-08 03:23:07,340] DEBUG [Producer clientId=console-producer] Initiating connection to node test.kafka:6667 (id: 1001 rack: null) (org.apache.kafka.clients.NetworkClient) >>[2023-01-08 03:23:07,340] DEBUG Set SASL client state to SEND_APIVERSIONS_REQUEST (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,340] DEBUG Creating SaslClient: client=kafka/test.kafka@TEST;service=kafka;serviceHostname=test.kafka;mechs=[GSSAPI] (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>>>>>>>>>>>[2023-01-08 03:23:07,341] DEBUG Added sensor with name node-1001.bytes-sent (org.apache.kafka.common.metrics.Metrics) >>>>>>>>>>>>[2023-01-08 03:23:07,342] DEBUG Added sensor with name node-1001.bytes-received (org.apache.kafka.common.metrics.Metrics) >>>>>>>[2023-01-08 03:23:07,343] DEBUG Added sensor with name node-1001.latency (org.apache.kafka.common.metrics.Metrics) >>>[2023-01-08 03:23:07,343] DEBUG [Producer clientId=console-producer] Created socket with SO_RCVBUF = 32768, SO_SNDBUF = 102400, SO_TIMEOUT = 0 to node 1001 (org.apache.kafka.common.network.Selector) >>>>[2023-01-08 03:23:07,344] DEBUG Set SASL client state to RECEIVE_APIVERSIONS_RESPONSE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >[2023-01-08 03:23:07,344] DEBUG [Producer clientId=console-producer] Completed connection to node 1001. Fetching API versions. (org.apache.kafka.clients.NetworkClient) >>>>>[2023-01-08 03:23:07,346] DEBUG Set SASL client state to SEND_HANDSHAKE_REQUEST (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>>>>[2023-01-08 03:23:07,347] DEBUG Set SASL client state to RECEIVE_HANDSHAKE_RESPONSE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) [2023-01-08 03:23:07,347] DEBUG Set SASL client state to INITIAL (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>>>>>[2023-01-08 03:23:07,349] DEBUG Set SASL client state to INTERMEDIATE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>>>>>>>>>>>>>>>>>>>>>>>>[2023-01-08 03:23:07,351] DEBUG Set SASL client state to CLIENT_COMPLETE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>>>>>>>>[2023-01-08 03:23:07,352] DEBUG Set SASL client state to COMPLETE (org.apache.kafka.common.security.authenticator.SaslClientAuthenticator) >>[2023-01-08 03:23:07,352] DEBUG [Producer clientId=console-producer] Initiating API versions fetch from node 1001. (org.apache.kafka.clients.NetworkClient) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>[2023-01-08 03:23:07,354] DEBUG [Producer clientId=console-producer] Recorded API versions for node 1001: (Produce(0): 0 to 6 [usable: 6], Fetch(1): 0 to 8 [usable: 8], ListOffsets(2): 0 to 3 [usable: 3], Metadata(3): 0 to 6 [usable: 6], LeaderAndIsr(4): 0 to 1 [usable: 1], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 4 [usable: 4], ControlledShutdown(7): 0 to 1 [usable: 1], OffsetCommit(8): 0 to 4 [usable: 4], OffsetFetch(9): 0 to 4 [usable: 4], FindCoordinator(10): 0 to 2 [usable: 2], JoinGroup(11): 0 to 3 [usable: 3], Heartbeat(12): 0 to 2 [usable: 2], LeaveGroup(13): 0 to 2 [usable: 2], SyncGroup(14): 0 to 2 [usable: 2], DescribeGroups(15): 0 to 2 [usable: 2], ListGroups(16): 0 to 2 [usable: 2], SaslHandshake(17): 0 to 1 [usable: 1], ApiVersions(18): 0 to 2 [usable: 2], CreateTopics(19): 0 to 3 [usable: 3], DeleteTopics(20): 0 to 2 [usable: 2], DeleteRecords(21): 0 to 1 [usable: 1], InitProducerId(22): 0 to 1 [usable: 1], OffsetForLeaderEpoch(23): 0 to 1 [usable: 1], AddPartitionsToTxn(24): 0 to 1 [usable: 1], AddOffsetsToTxn(25): 0 to 1 [usable: 1], EndTxn(26): 0 to 1 [usable: 1], WriteTxnMarkers(27): 0 [usable: 0], TxnOffsetCommit(28): 0 to 1 [usable: 1], DescribeAcls(29): 0 to 1 [usable: 1], CreateAcls(30): 0 to 1 [usable: 1], DeleteAcls(31): 0 to 1 [usable: 1], DescribeConfigs(32): 0 to 2 [usable: 2], AlterConfigs(33): 0 to 1 [usable: 1], AlterReplicaLogDirs(34): 0 to 1 [usable: 1], DescribeLogDirs(35): 0 to 1 [usable: 1], SaslAuthenticate(36): 0 [usable: 0], CreatePartitions(37): 0 to 1 [usable: 1], CreateDelegationToken(38): 0 to 1 [usable: 1], RenewDelegationToken(39): 0 to 1 [usable: 1], ExpireDelegationToken(40): 0 to 1 [usable: 1], DescribeDelegationToken(41): 0 to 1 [usable: 1], DeleteGroups(42): 0 to 1 [usable: 1]) (org.apache.kafka.clients.NetworkClient) >>[2023-01-08 03:23:07,357] DEBUG Added sensor with name topic.cleanCsv.records-per-batch (org.apache.kafka.common.metrics.Metrics) >>>>[2023-01-08 03:23:07,357] DEBUG Added sensor with name topic.cleanCsv.bytes (org.apache.kafka.common.metrics.Metrics) >>[2023-01-08 03:23:07,358] DEBUG Added sensor with name topic.cleanCsv.compression-rate (org.apache.kafka.common.metrics.Metrics) >[2023-01-08 03:23:07,358] DEBUG Added sensor with name topic.cleanCsv.record-retries (org.apache.kafka.common.metrics.Metrics) >>>[2023-01-08 03:23:07,358] DEBUG Added sensor with name topic.cleanCsv.record-errors (org.apache.kafka.common.metrics.Metrics) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>[2023-01-08 03:23:08,645] INFO [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer) [2023-01-08 03:23:08,645] DEBUG [Producer clientId=console-producer] Beginning shutdown of Kafka producer I/O thread, sending remaining records. (org.apache.kafka.clients.producer.internals.Sender) [2023-01-08 03:23:08,647] DEBUG Removed sensor with name connections-closed: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,647] DEBUG Removed sensor with name connections-created: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,647] DEBUG Removed sensor with name successful-authentication: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,647] DEBUG Removed sensor with name failed-authentication: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,648] DEBUG Removed sensor with name bytes-sent-received: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,648] DEBUG Removed sensor with name bytes-sent: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,648] DEBUG Removed sensor with name bytes-received: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,649] DEBUG Removed sensor with name select-time: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,649] DEBUG Removed sensor with name io-time: (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,649] DEBUG Removed sensor with name node--1.bytes-sent (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,649] DEBUG Removed sensor with name node--1.bytes-received (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,650] DEBUG Removed sensor with name node--1.latency (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,650] DEBUG Removed sensor with name node-1001.bytes-sent (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,650] DEBUG Removed sensor with name node-1001.bytes-received (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,650] DEBUG Removed sensor with name node-1001.latency (org.apache.kafka.common.metrics.Metrics) [2023-01-08 03:23:08,654] WARN [Principal=null]: TGT renewal thread has been interrupted and will exit. (org.apache.kafka.common.security.kerberos.KerberosLogin) [2023-01-08 03:23:08,654] DEBUG [Producer clientId=console-producer] Shutdown of Kafka producer I/O thread has completed. (org.apache.kafka.clients.producer.internals.Sender) [2023-01-08 03:23:08,655] DEBUG [Producer clientId=console-producer] Kafka producer has been closed (org.apache.kafka.clients.producer.KafkaProducer)
... View more
Labels:
11-17-2022
09:20 AM
I did all the suggestions and tried to follow all the steps, but when I am running command: ./kafka-console-producer.sh --broker-list host.kafka:6667 --topic cleanCsv --producer-property security.protocol=SASL_PLAINTEXT < /tmp/clean_csv_full.csv Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>[2022-11-17 17:06:35,693] WARN [Principal=null]: TGT renewal thread has been interrupted and will exit. (org.apache.kafka.common.security.kerberos.KerberosLogin) No issues with creation of topics or other, just when trying to push csv inside I am getting that error, when without Kerberos all goes good and smoothly uploads it. Any help is really very helpful, thank you in advance and looking forward to your reply
... View more