Member since
02-28-2022
144
Posts
13
Kudos Received
14
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
97 | 10-18-2024 12:29 PM | |
1885 | 09-05-2024 09:06 AM | |
2012 | 08-08-2024 05:10 AM | |
1752 | 05-16-2024 05:33 AM | |
553 | 04-22-2024 10:24 AM |
08-18-2022
11:19 AM
hello cloudera community, we identified that there is tablet_id with different volume among the cluster's tablet servers there is tablet_id with 700MB on one tablet server and the same tablet_id with 190MB on another tablet server how can we equalize this volumetry between tablet_id on all tablet servers we believe that with this different volumetry between the tablet_id, some hosts are using a lot of memory, practically 90% of memory of what was defined. 1 - we are using cloudera express 5.16.x 2 - a rebalance has already been performed on kudu recently
... View more
Labels:
- Labels:
-
Apache Kudu
-
Cloudera Manager
08-17-2022
10:52 AM
hi @Deepan_N the "kinit" command was successfully executed the "klist" command returns the date of the validated ticket and I have more than 10h to use the ticket. Both modes have been tested: curl: curl -v -u: --negotiate -X POST --data '{"className": "org.apache.spark.examples.SparkPi", "jars": ["/tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar"], "name": "livy-test", "file": "hdfs:///tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar", "args": [10]}' -H "Content-Type: application/json" -H "X-Requested-By: User" http://localhost:8998/batches python: import json, pprint, requests, textwrap from requests_kerberos import HTTPKerberosAuth host='http://localhost:8998' headers = {'Requested-By': 'livy','Content-Type': 'application/json','X-Requested-By': 'livy'} auth=HTTPKerberosAuth() data={'className': 'org.apache.spark.examples.SparkPi','jars': ["/tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar"],'name': 'livy-test1', 'file': 'hdfs:///tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar','args': ["10"]} r0 = requests.post(host + '/batches', data=json.dumps(data), headers=headers, auth=auth) r0.json() but unfortunately both ways return the error: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /batches. Reason: <pre> Authentication required</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/> </body> </html> PS¹: the cluster has kerberos PS²: the user that validated the ticket with "kinit" is the livy user that is created in AD (active directory)
... View more
08-16-2022
09:25 AM
hello cloudera community,
we have installed livy directly on a cloudera cluster host and we are now trying to connect to livy now using kerberos but we are getting the following error:
<html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /sessions. Reason: <pre> Authentication required</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/> </body> </html>
how can we make this connection to retest livy?
we are testing the connection with python3 script as follows:
import json, pprint, requests, textwrap
from requests_kerberos import HTTPKerberosAuth
host='http://localhost:8998'
data = {'kind': 'spark'}
headers = {'Requested-By': 'livy','Content-Type': 'application/json'}
auth=HTTPKerberosAuth()
r0 = requests.post(host + '/sessions', data=json.dumps(data), headers=headers,auth=auth)
r0.json()
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Cloudera Manager
08-10-2022
05:43 AM
good morning cloudera community, we are using apache ambari in version 2.6.2.2 and in this version we are using HDP 2.6.5, with that, we need to configure the parameter below in HDFS, but unfortunately we are not finding this option in the HDFS configuration, so we need to know how configure this parameter in HDFS? parameter: dfs.client.block.write.replace-datanode-on-failure.policy=ALWAYS PS: would it be in the "custom hdfs-site" option by clicking on "add property" and adding the parameter in the "properties" box?
... View more
Labels:
07-08-2022
08:46 AM
we are using kafka 2.2.1 in cloudera manager with cdh6.3.2 when we try to run kafka producer: kafka-console-producer --broker-list ip-172-31-13-30.us-east-2.compute.internal:9092,ip-172-31-12-151.us-east-2.compute.internal:9092,ip-172-31-14-136.us-east-2.compute.internal:9092 --topic test_topic returns the errors: 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-14-136.us-east-2.compute.internal:9092 (id: -3 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-12-151.us-east-2.compute.internal:9092 (id: -2 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-13-30.us-east-2.compute.internal:9092 (id: -1 rack: null) disconnected when we check the logs of kafka brokers there is: 2022-07-08 12:36:17,365 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=89] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,776 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=88] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,505 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=90] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) We would like to know why this problem is occurring? ps: we are managing to list the topics without problem
... View more
Labels:
- Labels:
-
Apache Kafka
-
Cloudera Manager
06-14-2022
01:42 PM
@MattWho / @bbahamondes I made alternate stops of the nifi nodes and with that I was able to access the nifi and configure the policies. Thank you for your help.
... View more
06-14-2022
12:37 PM
hi, @bbahamondes / @MattWho in the configuration "nifi.initial.admin.identity" in nifi, I left only the user "user.bind", stopped nifi, deleted the files "authorizations.xml" and "users.xml" and started nifi I don't know if it worked, but it's not giving the permission error anymore, but it's giving connection error on port 8080 when I try to login PS: in the configuration "Login Identity Provider: Default LDAP Identity Strategy" in the nifi is configured "USE_USERNAME" PS: when I try to login with a different user it returns the permission error: "Unknown user with identity 'user'. Contact the system administrator."
... View more
06-14-2022
10:58 AM
I'm configuring LDAP access (AD) on Nifi 1.0.1.0 being managed by Cloudera Manager, but when trying to access nifi with AD credentials, the error is returned: "Unknown user with identity 'user.bind'. Contact the system administrator." In Nifi's "nifi.initial.admin.identity" settings it looks like this: "cn=user.bind,ou=USERS,ou=CLOUDERA,dc=lab,dc=local" What could be wrong for not being able to access?
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera Manager
05-30-2022
06:16 AM
hi @rblough With the information given earlier, what could be causing this problem?
... View more
05-27-2022
07:28 AM
hi @rblough yes, the tablets server are balanced, today there are 1443 replicas, dividing by 5 tablet server gives a total of 288 replicas per tablet server, as shown by each tablet server below: hostname04.local - replicas by tablets - RUNNING-289 hostname05.local - replicas by tablets - RUNNING-288 hostname06.local - replicas by tablets - RUNNING-289 hostname07.local - replicas by tablets - RUNNING-289 hostname08.local - replicas by tablets - RUNNING-288 there have been times when host 06 was also used above 85% to 90% of memory usage. when we restart the tablet server that have high memory usage, there are times when the tablet server itself goes back up the memory usage or another tablet server that has low memory usage starts to go up, getting above 85 % too.
... View more