Member since
02-28-2022
147
Posts
13
Kudos Received
14
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
151 | 10-18-2024 12:29 PM | |
2930 | 09-05-2024 09:06 AM | |
2461 | 08-08-2024 05:10 AM | |
1919 | 05-16-2024 05:33 AM | |
651 | 04-22-2024 10:24 AM |
08-19-2022
08:00 AM
hi @Asfahan checking the file with the same name as tablet_id in the "consensus-meta" directory, it shows that the file is 11K in all TS as you can see in the screenshot below, the tablet_id has different sizes between the 3 TS
... View more
08-19-2022
06:26 AM
oi @Asfahan 1 - this parameter was defined in the kudu settings on cloudera: default_num_replicas = 3 2 - below is the result of the "fsck" command you asked for: command did not return tablet_id size on TS hosts
... View more
08-19-2022
05:51 AM
hi @Deepan_N by running the command below directly in python3: r0.headers["www-authenticate"] returns the following error: Python 3.6.8 (default, Nov 16 2020, 16:55:22) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> r0.headers["www-authenticate"] Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'r0' is not defined >>> below is the screenshot of the commands executed in bash:
... View more
08-18-2022
11:19 AM
hello cloudera community, we identified that there is tablet_id with different volume among the cluster's tablet servers there is tablet_id with 700MB on one tablet server and the same tablet_id with 190MB on another tablet server how can we equalize this volumetry between tablet_id on all tablet servers we believe that with this different volumetry between the tablet_id, some hosts are using a lot of memory, practically 90% of memory of what was defined. 1 - we are using cloudera express 5.16.x 2 - a rebalance has already been performed on kudu recently
... View more
Labels:
- Labels:
-
Apache Kudu
-
Cloudera Manager
08-17-2022
10:52 AM
hi @Deepan_N the "kinit" command was successfully executed the "klist" command returns the date of the validated ticket and I have more than 10h to use the ticket. Both modes have been tested: curl: curl -v -u: --negotiate -X POST --data '{"className": "org.apache.spark.examples.SparkPi", "jars": ["/tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar"], "name": "livy-test", "file": "hdfs:///tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar", "args": [10]}' -H "Content-Type: application/json" -H "X-Requested-By: User" http://localhost:8998/batches python: import json, pprint, requests, textwrap from requests_kerberos import HTTPKerberosAuth host='http://localhost:8998' headers = {'Requested-By': 'livy','Content-Type': 'application/json','X-Requested-By': 'livy'} auth=HTTPKerberosAuth() data={'className': 'org.apache.spark.examples.SparkPi','jars': ["/tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar"],'name': 'livy-test1', 'file': 'hdfs:///tmp/spark-examples-1.6.0-cdh5.16.1-hadoop2.6.0-cdh5.16.1.jar','args': ["10"]} r0 = requests.post(host + '/batches', data=json.dumps(data), headers=headers, auth=auth) r0.json() but unfortunately both ways return the error: <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /batches. Reason: <pre> Authentication required</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/> </body> </html> PS¹: the cluster has kerberos PS²: the user that validated the ticket with "kinit" is the livy user that is created in AD (active directory)
... View more
08-16-2022
09:25 AM
hello cloudera community,
we have installed livy directly on a cloudera cluster host and we are now trying to connect to livy now using kerberos but we are getting the following error:
<html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 401 </title> </head> <body> <h2>HTTP ERROR: 401</h2> <p>Problem accessing /sessions. Reason: <pre> Authentication required</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.24.v20180605</a><hr/> </body> </html>
how can we make this connection to retest livy?
we are testing the connection with python3 script as follows:
import json, pprint, requests, textwrap
from requests_kerberos import HTTPKerberosAuth
host='http://localhost:8998'
data = {'kind': 'spark'}
headers = {'Requested-By': 'livy','Content-Type': 'application/json'}
auth=HTTPKerberosAuth()
r0 = requests.post(host + '/sessions', data=json.dumps(data), headers=headers,auth=auth)
r0.json()
... View more
Labels:
- Labels:
-
Apache Hadoop
-
Cloudera Manager
08-10-2022
05:43 AM
good morning cloudera community, we are using apache ambari in version 2.6.2.2 and in this version we are using HDP 2.6.5, with that, we need to configure the parameter below in HDFS, but unfortunately we are not finding this option in the HDFS configuration, so we need to know how configure this parameter in HDFS? parameter: dfs.client.block.write.replace-datanode-on-failure.policy=ALWAYS PS: would it be in the "custom hdfs-site" option by clicking on "add property" and adding the parameter in the "properties" box?
... View more
Labels:
07-08-2022
08:46 AM
we are using kafka 2.2.1 in cloudera manager with cdh6.3.2 when we try to run kafka producer: kafka-console-producer --broker-list ip-172-31-13-30.us-east-2.compute.internal:9092,ip-172-31-12-151.us-east-2.compute.internal:9092,ip-172-31-14-136.us-east-2.compute.internal:9092 --topic test_topic returns the errors: 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-14-136.us-east-2.compute.internal:9092 (id: -3 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-12-151.us-east-2.compute.internal:9092 (id: -2 rack: null) disconnected 22/07/08 12:36:16 WARN clients.NetworkClient: [Producer clientId=console-producer] Bootstrap broker ip-172-31-13-30.us-east-2.compute.internal:9092 (id: -1 rack: null) disconnected when we check the logs of kafka brokers there is: 2022-07-08 12:36:17,365 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=89] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,776 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=88] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) 2022-07-08 12:34:11,505 INFO org.apache.kafka.common.network.Selector: [SocketServer brokerId=90] Failed authentication with /172.31.9.160 (Unexpected Kafka request of type METADATA during SASL handshake.) We would like to know why this problem is occurring? ps: we are managing to list the topics without problem
... View more
Labels:
- Labels:
-
Apache Kafka
-
Cloudera Manager
06-14-2022
01:42 PM
@MattWho / @bbahamondes I made alternate stops of the nifi nodes and with that I was able to access the nifi and configure the policies. Thank you for your help.
... View more
06-14-2022
12:37 PM
hi, @bbahamondes / @MattWho in the configuration "nifi.initial.admin.identity" in nifi, I left only the user "user.bind", stopped nifi, deleted the files "authorizations.xml" and "users.xml" and started nifi I don't know if it worked, but it's not giving the permission error anymore, but it's giving connection error on port 8080 when I try to login PS: in the configuration "Login Identity Provider: Default LDAP Identity Strategy" in the nifi is configured "USE_USERNAME" PS: when I try to login with a different user it returns the permission error: "Unknown user with identity 'user'. Contact the system administrator."
... View more