About yagoaparecidoti

yagoaparecidoti · ‎07-08-2022

we are using kafka 2.2.1 in cloudera manager with cdh6.3.2 when we try to run kafka producer: kafka-console-producer --broker-list ip-172-31-13-30.us-east-2.compute.internal:9092,ip-172-31-12-151.us-east-2.compute.internal:9092,ip-172-31-14-136.us-east-2.compute.internal:9092 --topic test_topic returns the errors: 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-14-136.us-east-2.compute.internal:9092 (id: -3 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-12-151.us-east-2.compute.internal:9092 (id: -2 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-13-30.us-east-2.compute.internal:9092 (id: -1 rack: null) disconnected when we check the logs of kafka brokers there is: 2022-07-08 12:36:17,365 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=89] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,776 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=88] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,505 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=90] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) We would like to know why this problem is occurring? ps: we are managing to list the topics without problem

yagoaparecidoti · ‎06-14-2022

@MattWho / @bbahamondes I made alternate stops of the nifi nodes and with that I was able to access the nifi and configure the policies. Thank you for your help.

yagoaparecidoti · ‎06-14-2022

hi, @bbahamondes / @MattWho in the configuration "nifi.initial.admin.identity" in nifi, I left only the user "user.bind", stopped nifi, deleted the files "authorizations.xml" and "users.xml" and started nifi I don't know if it worked, but it's not giving the permission error anymore, but it's giving connection error on port 8080 when I try to login PS: in the configuration "Login Identity Provider: Default LDAP Identity Strategy" in the nifi is configured "USE_USERNAME" PS: when I try to login with a different user it returns the permission error: "Unknown user with identity 'user'. Contact the system administrator."

yagoaparecidoti · ‎06-14-2022

I'm configuring LDAP access (AD) on Nifi 1.0.1.0 being managed by Cloudera Manager, but when trying to access nifi with AD credentials, the error is returned: "Unknown user with identity 'user.bind'. Contact the system administrator." In Nifi's "nifi.initial.admin.identity" settings it looks like this: "cn=user.bind,ou=USERS,ou=CLOUDERA,dc=lab,dc=local" What could be wrong for not being able to access?

yagoaparecidoti · ‎05-30-2022

hi @rblough With the information given earlier, what could be causing this problem?

yagoaparecidoti · ‎05-27-2022

hi @rblough yes, the tablets server are balanced, today there are 1443 replicas, dividing by 5 tablet server gives a total of 288 replicas per tablet server, as shown by each tablet server below: hostname04.local - replicas by tablets - RUNNING-289 hostname05.local - replicas by tablets - RUNNING-288 hostname06.local - replicas by tablets - RUNNING-289 hostname07.local - replicas by tablets - RUNNING-289 hostname08.local - replicas by tablets - RUNNING-288 there have been times when host 06 was also used above 85% to 90% of memory usage. when we restart the tablet server that have high memory usage, there are times when the tablet server itself goes back up the memory usage or another tablet server that has low memory usage starts to go up, getting above 85 % too.

yagoaparecidoti · ‎05-25-2022

hi @rblough 1) - answer: Masters | 3 Tablet Servers | 5 Tables | 48 Tablets | 481 Replicas | 1443 2) - answer: According to the chart "Total Tablet Size On Disk Across Kudu Replicas", it has more than 710GB 3) - answer: exist 5 4) - answer: exist 3 threads 5) - answer: memory_limit_hard_bytes = 60GB block_cache_capacity_mb = 15GB

yagoaparecidoti · ‎05-25-2022

kudu service has 3 kudu master and 5 kudu tablet among these 5 kudu tablet, 3 of them have high memory usage, 60GB has been set for each kudu tablet and it is using more than 85% to 90% of that memory and with that it ends up causing jobs failure below shows the memory usage behavior: TABLET SERVER: hostname04.local - USE MEMORY: 55.28G / %: 92.13% TABLET SERVER: hostname05.local - USE MEMORY: 54.22G / %: 90.38% TABLET SERVER: hostname06.local - USE MEMORY: 29.05G / %: 48.42% TABLET SERVER: hostname07.local - USE MEMORY: 28.47G / %: 47.46% TABLET SERVER: hostname08.local - USE MEMORY: 54.99G / %: 91.66% memory usage graph of all kudu tablet in cloudera manager: would like to understand why kudu is having this behavior and how can we reduce this memory usage? PS: kudu service is installed and being managed by cloudera manager express in version 5.16.2 and kudu service is in version 1.7

yagoaparecidoti · ‎05-24-2022

thank you @ask_bill_brooks I had seen this thread you passed, but it doesn't explicitly say which version of CM/CDH to run github's log4j script procedure. but I will consider version 5.x and 6.x to run this script.

yagoaparecidoti · ‎05-24-2022

Hello about log4j2 failure (CVE-2021-44228) in cloudera there is the KB: https://my.cloudera.com/knowledge/Resolution-for-TSB-2021-545---Critical-vulnerability-in-log4j2?id=332012 instructs to run the script to remove a class in the jar etc. our doubt is, do we need to run this script in any version of CM / CDH or just the version from 6.x.x?

Online	Offline
Last Visited	‎10-10-2025 07:05 AM

Member Since	‎02-28-2022 11:49 AM
Last Visited	‎10-10-2025 07:05 AM
Posts	171
Kudos received	14

Cloudera Community

Re: How to add Delay during Rolling Restart Until ...

Re: check date and time of access to some kudu tab...

Re: We need developer access to archive.cloudera.c...

Re: how to create a flow in nifi that reads a hive...

Re: how create, export and import full nifi templa...

Problem kafka producer - WARN clients.NetworkClien...

Re: NIFI - Unknown user with identity 'user'. Cont...

Re: NIFI - Unknown user with identity 'user'. Cont...

NIFI - Unknown user with identity 'user'. Contact ...

Re: kudu service is having high memory usage on sp...

Re: kudu service is having high memory usage on sp...

Re: kudu service is having high memory usage on sp...

kudu service is having high memory usage on specif...

Re: log4j2 failure (CVE-2021-44228) in cloudera

log4j2 failure (CVE-2021-44228) in cloudera