Member since
10-04-2017
113
Posts
11
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
10609 | 07-03-2019 08:34 AM | |
1591 | 10-31-2018 02:16 AM | |
9426 | 05-11-2018 01:31 AM | |
5928 | 02-21-2018 03:25 AM | |
2248 | 02-21-2018 01:18 AM |
06-29-2021
04:40 AM
I have upgraded from 7.1.1 to 7.1.6 and "Upgrade Ranger database and apply patches" is grayed out.
... View more
05-29-2020
07:14 AM
I did follow the similar steps but had the issue. I had to remove all the KTS/KMS installation start from scratch which fixed the issue but this time i added only one server first and then added the other.
... View more
04-16-2020
07:48 AM
I am enabling HDFS data at rest encryption in CDH 6.3 and while adding the KMS service, i noticed that /var/lib/kms-keytrustee/keytrustee/.keytrustee/ is actually doesn't exist on all the KMS hosts when trying to synchronise the KMS hosts private keys. Has anyone come across such issue and what is the way forward for this?
... View more
- Tags:
- cdh
- HDFS
- KEYTRUSTEE
- KMS
Labels:
04-06-2020
02:29 AM
@sbd4q0 I haven't tried it in ambari. Cloudera supports Hive metastore HA. So it was easy to add one more metastore and migrate as well.
... View more
12-17-2019
01:33 AM
@pdev have you disabled TLS in the management services as well? After you disable/enable TLS, you will have restart the management services so they communicate with CM with the changed port and protocol.
... View more
12-16-2019
08:43 AM
Hi,
We are planning to install Navigator encrypt in our cluster to encrypt kafka data and as part of this we would like to know if it is mandatory to setup TLS? We would also like to know the performance impact navigator encrypt has on kafka, does navigator encrypt have any logs to look at in case of any issue and can it be managed by cloudera manager.
Thanks,
Rajesh
... View more
Labels:
07-03-2019
08:34 AM
2 Kudos
@satz We were able to resolve this. We had the kerberos auth principles in default kafka group while all the broker were in a different config group. Adding the auth principles to the kafka config group has solved the issue.
... View more
06-09-2019
12:36 AM
Hi, We have just built a new kafka cluster( 3.1.0-1.3.1.0.p0.35 ) and integrated it with kerberos. Kerberos is integrated with AD. We are able to produce and consume with kafka principle which is local but with AD users, it fails with below error on console. 19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to CLIENT_COMPLETE 19/06/09 08:14:22 DEBUG authenticator.SaslClientAuthenticator: Set SASL client state to COMPLETE 19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Initiating API versions fetch from node -1. 19/06/09 08:14:22 DEBUG network.Selector: [Producer clientId=console-producer] Connection with server2.kafka4.corp/180.20.92.23 disconnected java.io.EOFException at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:124) at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:93) at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:235) at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:196) at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:545) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:483) at org.apache.kafka.common.network.Selector.poll(Selector.java:412) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:481) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:239) at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163) at java.lang.Thread.run(Thread.java:748) 19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Node -1 disconnected. 19/06/09 08:14:22 INFO clients.NetworkClient: [Producer clientId=console-producer] API versions request failed via disconnect. Defaulting legacy API versions 19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available 19/06/09 08:14:22 DEBUG clients.NetworkClient: [Producer clientId=console-producer] Give up sending metadata request since no node is available ^C19/06/09 08:14:22 INFO producer.KafkaProducer: [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. 19/06/09 08:14:22 DEBUG internals.Sender: [Producer clientId=console-producer] Beginning shutdown of Kafka producer I/O thread, sending remaining records. ========================================== The broker logs only have errors about the shortname for users. Caused by: org.apache.kafka.common.security.kerberos.NoMatchingRule: No rules apply to RAJESH@KAFKA4.CORP, rules [DEFAULT]
... View more
Labels:
- Labels:
-
Apache Kafka
-
Kerberos
05-29-2019
05:42 AM
If you install cloudera manager packages and start the agent, it will not distribute the parcels. The parcels will be distributed only when you add the host to the cluster.
... View more
05-29-2019
05:27 AM
Clouder amanager would automatically distribute the parcels that the cluster requires.
... View more
05-29-2019
04:31 AM
@Fawze You can mention multiple parcel links in cloudera manager so it should not be an issue.
... View more
04-05-2019
05:03 AM
@Harsh J We did try NFS didn't get the write performance we were looking at and more over that required an NFS server as well.
... View more
04-02-2019
08:34 AM
Hi , I'm using CDH 5.14 and have FUSE integrated. I'm having issue with the mounted HDFS sometimes( disconnecting and connecting ) and since FUSE doesn't have any log, its difficult to identify what is happening. Is it possible to start FUSE with logging? I do not see any logs in /var/log/messages regarding this.
... View more
Labels:
- Labels:
-
HDFS
11-28-2018
02:35 AM
Hi @manuroman I have read the page but couldn't validate myself on how many nodes are actually under billing. The License page in CM shows below and makes sense as we have 10 gateways and they are excluded in the total below. Is the billing calculated based on how many nodes have the core product or based on how many nodes are under navogator? Total Nodes : 101
... View more
11-28-2018
02:08 AM
Hi, How do we know how many nodes in the cluster are used for licensing? Like edge nodes are not used for billing etc.,. We have clusters in CDH 5.5.4 and CDH 5.14.2.
... View more
Labels:
- Labels:
-
Cloudera Manager
11-27-2018
02:54 AM
1 Kudo
CFile is an on-disk columnar storage format which holds data and associated B-Tree indexes. https://github.com/cloudera/kudu/blob/master/docs/design-docs/cfile.md
... View more
11-27-2018
02:51 AM
Hi , I it possible to set idle_query_timeout in impala shell in CDH 5.14.2? The documentation in below link tells that "You can override the --idle_session_timeout value with the IDLE_SESSION_TIMEOUT Query Option (CDH 5.15 / Impala 2.12 or higher only) at the session level". Do we have any other option in CDH 5.14.2? https://www.cloudera.com/documentation/enterprise/latest/topics/impala_timeouts.html
... View more
Labels:
- Labels:
-
Apache Impala
10-31-2018
02:16 AM
This is because navigator upgrade generally takes lot of time based on the number of objects and relations you have. Increasing the navigator heap size can help. Calculation of the required heap is available in cloudera site.
... View more
10-27-2018
12:37 PM
1 Kudo
Hi, We often see NOT_LEADER_FOR_PARTITION in kafka. On what basis does kafka leader election happen ? What are all the reasons that cause kafka partition leader election to trigger?
... View more
Labels:
- Labels:
-
Apache Kafka
09-12-2018
02:43 AM
@Harsh J Thanks for the detailed answer. The cluster we have is having very less data and all the datanodes haven't utilized atleast 10% of storage yet but is expected to use atleast 60% in a week. Would running HDFS balancer and compaction make any difference in this case!
... View more
09-11-2018
10:55 AM
1 Kudo
Hi, What is the procedure to follow when addin additional region server to an existing cluster? We use CDH 5.14
... View more
Labels:
- Labels:
-
Apache HBase
-
Cloudera Manager
08-31-2018
06:12 AM
Hi, We have recently upgraded from 5.11.2 to 5.14.2. The upgrade was untill we identified that many databases were not visible/loaded in navigator. The logs have below. Any related pointers are welcome. 2018-08-31 10:12:17,344 ERROR com.cloudera.nav.hive.extractor.AbstractHiveExtractor [CDHExecutor-0-CDHUrlClassLoader@7fbf5fdd]: Failed to extract database dummy_database with error: java.net.SocketException: Broken pipe (Write failed) org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed) at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) at org.apache.thrift.transport.TTransport.write(TTransport.java:107) at org.apache.thrift.transport.TSaslTransport.writeLength(TSaslTransport.java:391) at org.apache.thrift.transport.TSaslTransport.flush(TSaslTransport.java:499) at org.apache.thrift.transport.TSaslClientTransport.flush(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.TFilterTransport.flush(TFilterTransport.java:77) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_database(ThriftHiveMetastore.java:664) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:656) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:1213) at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.extractDatabase(AbstractHiveExtractor.java:147) at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.extractDatabases(AbstractHiveExtractor.java:133) at com.cloudera.nav.hive.extractor.HiveExtractor.run(HiveExtractor.java:63) at com.cloudera.nav.hive.extractor.AbstractHiveExtractor.run(AbstractHiveExtractor.java:118) at com.cloudera.nav.hive.extractor.HiveExtractorShim.run(HiveExtractorShim.java:35) at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:221) at com.cloudera.cmf.cdhclient.CdhExecutor$RunnableWrapper.call(CdhExecutor.java:211) at com.cloudera.cmf.cdhclient.CdhExecutor$CallableWrapper.doWork(CdhExecutor.java:236) at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper$1.run(CdhExecutor.java:189) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) at com.cloudera.cmf.cdh5client.security.UserGroupInformationImpl.doAs(UserGroupInformationImpl.java:44) at com.cloudera.cmf.cdhclient.CdhExecutor$SecurityWrapper.doWork(CdhExecutor.java:186) at com.cloudera.cmf.cdhclient.CdhExecutor$1.call(CdhExecutor.java:125) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) Caused by: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) Thanks
... View more
Labels:
- Labels:
-
Apache Hive
-
Cloudera Navigator
08-30-2018
02:02 AM
Hi, Try only below in your consumer properties file. security.protocol=SASL_PLAINTEXT sasl.mechanism=GSSAPI sasl.kerberos.service.name=kafka group.id=testgroup
... View more
07-04-2018
03:46 AM
Hi @bgooley We have users in multiple domins that want access to navigator UI but the configurations allow only one AD domain and there is no option in Navigator to add individual users as well if we have to add local users instead of AD users. The option in cloudera manager provides navigator admin access to local users while we require read-only in navigator.
... View more
07-03-2018
06:10 AM
Hi, Yes it was infact the java version issue and compatability between CM5.5 and Centos 7.4. Had to tweek client_configs.py to make it work.
... View more
07-03-2018
06:08 AM
Hi, We are using CM 5.14 and CDH 5.14. We have configured Hue to use multiple AD domains but we do not have any info on how it can be done on Cloudera navigator. Any pointers are appreciated. Thanks
... View more
Labels:
06-28-2018
10:51 AM
Hi, Our cluster is in RHEL 6 and i'm trying to add a new gateway that is on CentOS7. I have manually inatalled the agents but starting the agent shows below error. And adding the host to cluster gets stuck in activating the parcel. [28/Jun/2018 18:49:04 +0000] 25958 MainThread downloader INFO Downloader path: /opt/cloudera/parcel-cache [28/Jun/2018 18:49:04 +0000] 25958 MainThread parcel_cache INFO Using /opt/cloudera/parcel-cache for parcel cache [28/Jun/2018 18:49:04 +0000] 25958 MainThread agent ERROR Caught unexpected exception in main loop. Traceback (most recent call last): File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 578, in start self._init_after_first_heartbeat_response(heartbeat_response["data"]) File "/usr/lib64/cmf/agent/src/cmf/agent.py", line 700, in _init_after_first_heartbeat_response self.client_configs.load() File "/usr/lib64/cmf/agent/src/cmf/client_configs.py", line 682, in load new_deployed.update(self._lookup_alternatives(fname)) File "/usr/lib64/cmf/agent/src/cmf/client_configs.py", line 432, in _lookup_alternatives return self._parse_alternatives(alt_name, out) File "/usr/lib64/cmf/agent/src/cmf/client_configs.py", line 444, in _parse_alternatives path, _, _, priority_str = line.rstrip().split(" ") ValueError: too many values to unpack ^C
... View more
Labels:
- Labels:
-
Cloudera Manager
06-21-2018
01:30 AM
Hi, We have a CDH cluster in 5.14.2. Is it compatable with KTS 3.8.x ? Due to some limitations, we are unable to use KTS 5.14.0 in this cluster. The cloudera documentation shows KTS compatability with KMS and CM but nor CDH.
... View more
Labels:
- Labels:
-
Cloudera Navigator
05-25-2018
09:02 AM
Hi, I'm using cloudera manager 5.14.3 and CDK 3.0.0. Rigth now we use CLI to monitor the consumer group lag and I would like to monitor consumer group lag from cloudera manager. Is it possible?
... View more
Labels:
- Labels:
-
Apache Kafka
-
Cloudera Manager