About cloude

Mounika_Reddy · ‎09-11-2024

Hi @Marks_08 In Hive is HBase dependency is enabled? If No, can you follow the below doc https://docs.cloudera.com/cdp-private-cloud-base/7.1.6/accessing-hbase/topics/hbase-configure-hive-hbase-pvc.html If yes, we require the complete HS2 logs as well as the Yarn application logs to troubleshoot further. Along with that share us the DDL for both Managed and external tables DESCRIBE FORMATTED <TABLE_NAME>; SHOW CREATE TABLE <TABLE_NAME>; Replace TABLE_NAME with the managed and external tables. Thank You.

JiHoone · ‎08-06-2024

Hi @cloude, I changed 'ranger.usersync.ldap.referral' from 'ignore' to 'follow'. But it still doesn't work.. Thanks.

TMauran · ‎08-05-2024

Hello @cloude even after adding those parameters I get the 302 error, wondering if the issue is not on the Knox part though now but thank you very much for your answer

VidyaSargur · ‎01-17-2024

@Nardoleo, Did any of the responses assist in resolving your query? If it did, kindly mark the relevant reply as the solution, as it will aid others in locating the answer more easily in the future.

cloude · ‎01-09-2024

@SylvainL It seems your API request has some issues, Can you share the full request you are sending to /flow-files?

cloude · ‎01-06-2024

@phir1 , Create a queue for each dept and assign the requested resources. Now you know the dept is using how much resources.

jayes · ‎12-26-2023

Hello @smruti , Please let me know if any way where we can have call or meet to review the script ? I will in mean time will try to remove the -n options and check if that works.

vec · ‎12-21-2023

I deployed 3 zookeeper nodes and they are running well. And zK logs don't print any errors. After I stopped all zk nodes . The RM log prints below errors: 2023-12-22 07:34:11,542 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server cdp3.oia.com/192.168.1.176:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2023-12-22 07:34:11,542 INFO org.apache.zookeeper.ClientCnxn: Socket error occurred: cdp3.oia.com/192.168.1.176:2181: Connection refused 2023-12-22 07:34:11,642 WARN org.apache.zookeeper.Login: TGT renewal thread has been interrupted and will exit. 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.Login: Client successfully logged in. 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism. 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.Login: TGT refresh thread started. 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.Login: TGT valid starting at: Fri Dec 22 07:34:10 CST 2023 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.Login: TGT expires: Sat Dec 23 07:34:10 CST 2023 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.Login: TGT refresh sleeping until: Sat Dec 23 03:31:02 CST 2023 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server cdp1.oia.com/192.168.1.205:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2023-12-22 07:34:11,645 INFO org.apache.zookeeper.ClientCnxn: Socket error occurred: cdp1.oia.com/192.168.1.205:2181: Connection refused 2023-12-22 07:34:12,746 WARN org.apache.zookeeper.Login: TGT renewal thread has been interrupted and will exit. 2023-12-22 07:34:12,748 INFO org.apache.zookeeper.Login: Client successfully logged in. 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism. 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.Login: TGT refresh thread started. 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.Login: TGT valid starting at: Fri Dec 22 07:34:11 CST 2023 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.Login: TGT expires: Sat Dec 23 07:34:11 CST 2023 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.Login: TGT refresh sleeping until: Sat Dec 23 03:33:14 CST 2023 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server cdp2.oia.com/192.168.1.169:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2023-12-22 07:34:12,749 INFO org.apache.zookeeper.ClientCnxn: Socket error occurred: cdp2.oia.com/192.168.1.169:2181: Connection refused Resumed the zk nodes , it looks connection established but throws an exception in the end. 2023-12-22 07:34:33,729 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server cdp2.oia.com/192.168.1.169:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2023-12-22 07:34:33,730 INFO org.apache.zookeeper.ClientCnxn: Socket connection established, initiating session, client: /192.168.1.205:56000, server: cdp2.oia.com/192.168.1.169:2181 2023-12-22 07:34:33,757 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server cdp2.oia.com/192.168.1.169:2181, sessionid = 0x2002158fc320000, negotiated timeout = 40000 2023-12-22 07:34:33,758 INFO org.apache.curator.framework.state.ConnectionStateManager: State change: CONNECTED 2023-12-22 07:34:33,781 INFO org.apache.curator.framework.imps.EnsembleTracker: New config event received: {server.1=cdp1.oia.com:3181:4181:participant, version=0, server.3=cdp3.oia.com:3181:4181:participant, server.2=cdp2.oia.com:3181:4181:participant} 2023-12-22 07:34:33,784 INFO org.apache.curator.framework.imps.EnsembleTracker: New config event received: {server.1=cdp1.oia.com:3181:4181:participant, version=0, server.3=cdp3.oia.com:3181:4181:participant, server.2=cdp2.oia.com:3181:4181:participant} 2023-12-22 07:34:33,795 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /confstore/CONF_STORE at org.apache.zookeeper.KeeperException.create(KeeperException.java:120) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:1793) at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:274) at org.apache.curator.framework.imps.DeleteBuilderImpl$5.call(DeleteBuilderImpl.java:268) at org.apache.curator.connection.StandardConnectionHandlingPolicy.callWithRetry(StandardConnectionHandlingPolicy.java:67) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:81) at org.apache.curator.framework.imps.DeleteBuilderImpl.pathInForeground(DeleteBuilderImpl.java:265) at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:249) at org.apache.curator.framework.imps.DeleteBuilderImpl.forPath(DeleteBuilderImpl.java:34) at org.apache.hadoop.util.curator.ZKCuratorManager.delete(ZKCuratorManager.java:331) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.ZKConfigurationStore.format(ZKConfigurationStore.java:148) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.deleteRMConfStore(ResourceManager.java:1658) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1534) 2023-12-22 07:34:33,803 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down ResourceManager at cdp1.oia.com/192.168.1.205 ************************************************************/ The workaround I use is that I deleted the zk , yarn queue manager and yarn . And redeployed yarn only . so far it looks good.

cloude · ‎08-03-2022

Hi All, Below are the steps to setup load balancer on top of multiple Knox gateways. I will assume that you have KnoxSSO already set and services are going through it. Install knox gateway on another node Ambari/CM -> Hosts -> Select any host(where you want to install knox) and add knox gateway. Installation of HAproxy and Rsyslog: yum install -y haproxy rsyslog Step 3: Update the Rsyslog for checking the haproxy logs /etc/rsyslog.conf: ## Uncomment below line $ModLoad imudp $UDPServerRun 514 ##Add following directive in the same file. # HAProxy Logging local2.* /var/log/haproxy.log Create SSL certificate for haproxy: openssl genrsa -out /etc/haproxy/haproxy.key 2048 openssl req -new -key /etc/haproxy/haproxy.key -out /etc/haproxy/haproxy.csr -subj "/C=US/ST=North Carolina/L=Raleigh/O=HWX/OU=Support/CN=$(hostname -f)" openssl x509 -req -days 365 -in /etc/haproxy/haproxy.csr -signkey /etc/haproxy/haproxy.key -out /etc/haproxy/haproxy.crt cat /etc/haproxy/haproxy.key /etc/haproxy/haproxy.crt > /etc/haproxy/haproxy.pem Update haproxy configuration file /etc/haproxy/haproxy.cfg with both Knox hosts and LB server, In my case c2110-node1 and c2110-node2 are Knox gateway host and c2110-node3 is the LB node: #--------------------------------------------------------------------- # main frontend which proxys to the backends #--------------------------------------------------------------------- frontend knox-frontend-443 mode http option httplog bind *:443 ssl crt /etc/haproxy/haproxy.pem option forwardfor http-request redirect location https://%[req.hdr(Host)]/gateway/homepage/home/ if { path / } default_backend knox-backend-8443 backend knox-backend-8443 mode http option httplog option forwardfor balance roundrobin stick-table type ip size 1m expire 24h stick on src option httpchk HEAD /gateway/knoxsso/knoxauth/login.html HTTP/1.1\r\nHost:\ c1110-node1:8443 http-check expect status 200 server knox1 c1110-node2:8443 check ssl verify none server knox2 c1110-node3:8443 check ssl verify none Start the services: systemctl start haproxy systemctl enable haproxy systemctl start rsyslog.service set LB url using ambari-server setup-sso command, Only change the url as PEM is already setup for knox1 hosts: [root@c2110-node1 ~]# ambari-server setup-sso Using python /usr/bin/python Setting up SSO authentication properties... Enter Ambari Admin login: admin Enter Ambari Admin password: SSO is currently enabled Do you want to disable SSO authentication [y/n] (n)? Provider URL (https://c2110-node3.squadron.support.hortonworks.com:8443/gateway/knoxsso/api/v1/websso): https://c2110-node3.squadron.support.hortonworks.com:443/gateway/knoxsso/api/v1/websso The SSO provider's public certificate has already set. Do you want to change it [y/n] (n)? Use SSO for Ambari [y/n] (n)? Manage SSO configurations for eligible services [y/n] (y)? Use SSO for all services [y/n] (y)? JWT Cookie name (hadoop-jwt): JWT audiences list (comma-separated), empty for any (): Ambari Server 'setup-sso' completed successfully. Generate the gateway-identity file and copy on both nodes: # will create gateway.jks in keystore folder $GATEWAY_HOME/bin/knoxcli.sh create-cert --hostname localhost # to check the CN=localhost keytool -v -list -keystore gateway.jks # create the gateway-identity.pem certificate from gateway.jks file keytool -exportcert -alias gateway-identity -rfc -file $certificate_path -keystore $gateway_home/data/security/keystores/gateway.jks # to check the CN=localhost openssl x509 -in gateway-identity.pem -text # scp these two files from one node to another scp /usr/hdp/3.1.0.0-78/knox/data/security/keystores/gateway* c2110-node2:/usr/hdp/3.1.0.0-78/knox/data/security/keystores/ Thanks for reading.

Online	Offline
Last Visited	‎12-05-2024 02:58 AM

Member Since	‎01-22-2019 07:40 PM
Last Visited	‎12-05-2024 02:58 AM
Posts	17
Kudos received	6

Cloudera Community

Re: Storage and compute analysis

Re: Issue trying to insert into Managed Table from...

Re: Ranger users who were synced from AD aren't ma...

Re: Spark History CORS header ‘Access-Control-Allo...

Re: Delay with Spark application

Re: NIFI REST API- FlowFile exceeds maximum numbe...

Re: Storage and compute analysis

Re: Beeline command failing with error 4

Re: ResourceManager failed to start.

Knox HA / Loadbalancing using Haproxy