Member since
09-27-2019
18
Posts
0
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
1213 | 12-12-2019 03:28 AM |
07-22-2020
07:21 AM
Thanks a lot, Paras. pdev
... View more
07-22-2020
06:24 AM
Thanks, Paras will try and let you know. One more question, do BDR works between two different CDH versions? like 5.14 and 5.16 ? pdev
... View more
07-21-2020
06:57 AM
Hi,
I am running cdh 5.14.0 cluster version both on Prod and DR cluster. Both cluster are Kerberos and TLS enabled with the same kdc server.
Could anyone guide me on how to do BDR between Kerberos enabled clusters. As I was not able to find any proper guide on the Cloudera website.
Please help me or provide any docs related to this.
Appreciate your help.
Thanks
pdev
... View more
Labels:
03-06-2020
10:35 PM
Hi, Thanks for the reply.Actually we do not have any DNS.But when this error came,then for cross verification i Checked all principles in KDC server. In those principles , zookeeper service principal was absent. Dont know the reason,whycloudera manager was not able to create principal only for zookeeper. Waiting for your response. Thanks Pdev
... View more
03-06-2020
06:29 AM
Hello,
We are running CDH 5.14.0.We were setting up kerberos on our cluster.For this we were trying with MIT kerberos.Please see the Krb5.conf and kdc.conf files details as below.
krb5.conf:
]$ cat /etc/krb5.conf [libdefaults] default_realm = HADOOP.COM dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5 permitted_enctypes = aes256-cts-hmac-sha1-96 aes128-cts-hmac-sha1-96 arcfour-hmac-md5
[realms] HADOOP.COM = { kdc = hadoop-admin admin_server = hadoop-admin }
kdc.conf:
sudo cat /var/kerberos/krb5kdc/kdc.conf [kdcdefaults] kdc_ports = 88 kdc_tcp_ports = 88
[realms] HADOOP.COM = { #master_key_type = aes256-cts acl_file = /var/kerberos/krb5kdc/kadm5.acl dict_file = /usr/share/dict/words admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab supported_enctypes = aes256-cts-hmac-sha1-96:normal aes128-cts-hmac-sha1-96:normal des3-hmac-sha1:normal arcfour-hmac-md5:normal max_renewable_life = 7d }
During setup it failed with zookeeper error.We checked the error logs for zookeeper.Please find error :
[root@hadoop-slave-1 ~]# tail -f /var/log/zookeeper/zookeeper-cmf-zookeeper-SERVER-hadoop-slave-1.log 2020-03-06 17:08:16,659 INFO org.apache.zookeeper.server.DatadirCleanupManager: Purge task started. 2020-03-06 17:08:16,672 INFO org.apache.zookeeper.server.quorum.QuorumPeerMain: Starting quorum peer 2020-03-06 17:08:16,676 INFO org.apache.zookeeper.server.DatadirCleanupManager: Purge task completed. 2020-03-06 17:08:16,768 ERROR org.apache.zookeeper.server.quorum.QuorumPeerMain: Unexpected exception, exiting abnormally java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth. login.LoginException: Client not found in Kerberos database (6) - CLIENT_NOT_FOUND at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:207) at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)
Do anybody have an idea on that.Please share if you have solution.
Thanks
pdev
... View more
Labels:
- Labels:
-
Apache Zookeeper
-
Kerberos
02-03-2020
06:23 AM
Hi,
We are using cdh 5.14.0. We are running below services :
Flume HBase HDFS Hive Hue Kafka Oozie Sentry Spark YARN (MR2 Included) ZooKeeper
Can anyone help us to understand ,where important config files for the above-mentioned
services are saved?
Thanks
pdev
... View more
Labels:
- Labels:
-
Cloudera Manager
01-20-2020
03:12 AM
Hi,
We are running cdh 5.14.0 version cluster.Actually we were trying to install kafka parcels through local repository.We have downloaded the parcels and kept it under local repo dir and created hash file also.
[root@hadoop-admin cdh5.14.0]# pwd /var/www/html/cdh5.14.0 [root@hadoop-admin cdh5.14.0]# ls -l total 1878248 -rwxr-xr-x. 1 root root 1849867708 Aug 23 20:21 CDH-5.14.0-1.cdh5.14.0.p0.24-el6.parcel -rwx------. 1 cloudera-scm cloudera-scm 73369070 Jan 20 12:29 KAFKA-3.1.0-1.3.1.0.p0.35-el6.parcel -rw-r--r--. 1 cloudera-scm cloudera-scm 41 Jan 20 16:14 KAFKA-3.1.0-1.3.1.0.p0.35-el6.parcel.sha -rwx------. 1 root root 74072 Aug 26 17:08 manifest.json
After that we had set local repo path under cloudera manager.
but after that when checking for new parcels ,then this error is coming.
Error for parcel KAFKA-3.1.0-1.3.1.0.p0.35-el6.parcel : No valid hash file found.
Do anyone have any idea on this.Please share.
Thanks
pdev
... View more
Labels:
- Labels:
-
Apache Kafka
-
Cloudera Manager
12-19-2019
12:24 AM
Hi,
We are facing below error for hive metaserver. Please find the logs.
1:21:02.464 PM ERROR RetryClientInvocationHandler [pool-5-thread-47]: failed to execute syncNotifications java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606)
1:21:02.465 PM ERROR SentrySyncHMSNotificationsPostEventListener [pool-5-thread-47]: Failed to sync requested HMS notifications up to the event ID: 35354 java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy27.syncNotifications(Unknown Source) at org.apache.sentry.binding.metastore.SentrySyncHMSNotificationsPostEventListener.syncNotificationEvents(SentrySyncHMSNotificationsPostEventListener.java:153) at org.apache.sentry.binding.metastore.SentrySyncHMSNotificationsPostEventListener.onCreateDatabase(SentrySyncHMSNotificationsPostEventListener.java:108)
Does anyone have idea on that.Please share.
Thanks.
pdev
... View more
- Tags:
- Hive
- Metastore canary
Labels:
- Labels:
-
Apache Hive
12-18-2019
02:56 AM
thanks a lot Li Wang.It worked . pdev
... View more
12-17-2019
02:21 AM
Hi Rajesh, We have disabled tls from cloudera admin and removed truststore path and password from cloudera management services. I hope this what you are asking.if not then please share steps to disable it. Thanks. Pdev
... View more
12-17-2019
12:02 AM
Hi,
We were trying to enable TLS on our cluster.We have followed cloudera steps ,but i think missed something.After enabling TLS,we were able to login to port 7183,but all cloudera management services were disabled.
So we reverted back TLS changes from cloudera and removed all keys and certificates.After that given restart to the cloudera manager.
After this process,port 7180 started and login page came.But when i am trying to login then it is not redirecting to the GUI.
when i have check the logs. then these exceptions are showing.
2019-12-17 13:24:29,320 INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allow job details for all jobs. 2019-12-17 13:24:30,850 WARN ActionablesProvider-0:com.cloudera.server.web.cmf.StatusProvider: Failed to submit task for getting status from SERVICE_MONITORING com.cloudera.cmon.MgmtServiceNotRunningException: SERVICE_MONITORING is not running
2019-12-17 13:24:50,238 WARN EventStorePublisherWithRetry-0:com.cloudera.cmf.event.publish.EventStorePublisherWithRetry: Failed to publish event: SimpleEvent{attributes={CATEGORY=[AUDIT_EVENT], SEVERITY=[INFORMATIONAL], SERVICE=[ClouderaManager], SERVICE_TYPE=[ManagerServer], USER=[admin], EVENTCODE=[EV_LOGIN_SUCCESS], MESSAGE_CODES=[LOGIN_SUCCESS]}, content=User admin logged in successfully., timestamp=1576569290230} 2019-12-17 13:24:52,250 INFO 1267052060@scm-web-0:com.cloudera.server.web.cmf.AuthenticationFailureEventListener: Authentication failure for user: '' from 172.26.194.114 2019-12-17 13:25:05,877 INFO 1267052060@scm-web-0:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from 172.26.194.114 2019-12-17 13:25:11,032 INFO CMMetricsForwarder-0:com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder: Failed to send metrics. java.lang.reflect.UndeclaredThrowableException at com.sun.proxy.$Proxy122.writeMetrics(Unknown Source) at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendWithAvro(ClouderaManagerMetricsForwarder.java:325) at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.sendMetrics(ClouderaManagerMetricsForwarder.java:312) at com.cloudera.server.cmf.components.ClouderaManagerMetricsForwarder.run(ClouderaManagerMetricsForwarder.java:146) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.avro.AvroRemoteException: java.net.ConnectException: Connection refused at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:88) ... 11 more Caused by: java.net.ConnectException: Connection refused
Do anyone have any idea on that.Please suggest.
Thanks.
Devendra
... View more
Labels:
- Labels:
-
Cloudera Manager
12-05-2019
11:05 PM
Hi,
We are running CDH 5.14.0 version of cloudera.There are multiple warnings coming related to log data directory for only one Data node( hadoop-slave-1). please see warnings.
for the same node,root / dir space is showing critical.
can anyone suggest what files i should delete from "/ ,so that i can clear these alarms .
Thanks
Devendra
... View more
Labels:
- Labels:
-
Cloudera Manager
09-27-2019
07:21 AM
Hi,
We are using CDH 5.14. Webhdfs is enabled by default on cloudera manger and i am able to list HDFS data through webhdfs protocol.
[root@hadoop-slave-1 ~]# curl -i -L "http://172.16.0.10:50070/webhdfs/v1/?op=LISTSTATUS" HTTP/1.1 200 OK Cache-Control: no-cache Expires: Fri, 27 Sep 2019 13:52:56 GMT Date: Fri, 27 Sep 2019 13:52:56 GMT Pragma: no-cache Expires: Fri, 27 Sep 2019 13:52:56 GMT Date: Fri, 27 Sep 2019 13:52:56 GMT Pragma: no-cache Content-Type: application/json X-FRAME-OPTIONS: SAMEORIGIN Transfer-Encoding: chunked
{"FileStatuses":{"FileStatus":[ {"accessTime":0,"blockSize":0,"childrenNum":2,"fileId":33381,"group":"supergroup","length":0,"modificationTime":1569326622576,"owner":"hdfs","pathSuffix":"cli","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":10,"fileId":17365,"group":"supergroup","length":0,"modificationTime":1569589022824,"owner":"hdfs","pathSuffix":"hadoop","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":5,"fileId":33483,"group":"supergroup","length":0,"modificationTime":1569331210359,"owner":"hdfs","pathSuffix":"restorefromcli","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":33489,"group":"supergroup","length":0,"modificationTime":1569331240068,"owner":"hdfs","pathSuffix":"restorefromcli1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":4,"fileId":33645,"group":"supergroup","length":0,"modificationTime":1569331744422,"owner":"hdfs","pathSuffix":"restorefromcli_1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":10,"fileId":33346,"group":"supergroup","length":0,"modificationTime":1569589022883,"owner":"hdfs","pathSuffix":"restorehdfs","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":3,"fileId":16386,"group":"supergroup","length":0,"modificationTime":1568894436274,"owner":"hdfs","pathSuffix":"tmp","permission":"1777","replication":0,"storagePolicy":0,"type":"DIRECTORY"}, {"accessTime":0,"blockSize":0,"childrenNum":5,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1569324647144,"owner":"hdfs","pathSuffix":"user","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
But i am not able to see any config related to webhdfs in hdfs-site.xml .My concern is that, if this value( dfs.webhdfs.enabled) is enabled in cloudera, then it should also reflect in hdfs-site.xml file.
do anybody have any idea?
Thanks
Devendra
... View more
Labels:
- Labels:
-
Cloudera Manager