Cloudera Community
vladislav_falfu
user rank
Expert Contributor

Re: Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎02-07-2017 10:32 AM
‎02-07-2017 10:32 AM
Completed with some basic puppet setup. It required rewriting of some code from https://forge.puppet.com/unibet/ambari. Also found out that following puppet configuration does not support ssl. Trying to implement it, but had not found any manual how to configure ssl for ambari not using "ambari-server setup-security" as specified here https://community.hortonworks.com/articles/39865/enabling-https-for-ambariserver-and-troubleshootin.html. I assume that only options needed to make ssl working are: 1) Keystore for ambari services: " ssl.trustStore.path=<path-to-keystore-file> ssl.trustStore.type=jks ssl.trustStore.password=<keystore-password-here> " 2) HTTPS options: " client.api.ssl.port=8443 api.ssl=true client.api.ssl.cert_name=https.crt client.api.ssl.key_name=https.key " Isn`t it simpler to use nginx with ssl in front of ambari to provide https? Just tried to make a manual setup with generating certificates and filling the needed fields in /etc/ambari-server/conf/ambari.properties, afterwards started "ambari-server setup -s", but got "ambari-server restart" failed with error about problem with certificates and truststore. Seems it makes some transformations in process of running setup. Found out that there are some options for ambari-server setup-security option: 1) Configure https: "ambari-server setup-security --security-option=setup-https --api-ssl=true --import-cert-path=/etc/ambari-server/certs/localhost.crt --import-key-path=/etc/ambari-server/certs/localhost.key --pem-password=hadoop --api-ssl-port=8443" 2) Configure trustedstore: "ambari-server setup-security --security-option=setup-truststore --truststore-reconfigure --truststore-type=jks --truststore-password=hadoop --truststore-path=/etc/ambari-server/certs/ambari-server-truststore" Still looking into possibility of puppet-friendly setup. ... View more

Re: Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎02-01-2017 10:17 AM
‎02-01-2017 10:17 AM
Faced some question of consistency. Is this configuration https://forge.puppet.com/unibet/ambari fully consistent with latest ambari version? ... View more

Re: Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎01-31-2017 10:12 AM
‎01-31-2017 10:12 AM
Faced several issues when making the deployment: 1. Ambari puppet configurations that can be found on puppetforge are tested only for CentOS and still some features are not supported. I am using debian. Trying to test and make some rewrites for debian and fix some issues. 2. It would be nice to have some utility for managing blueprints (something similar to StackMaster for managing Amazon CloudFormation stacks). Had been thinking about such utility, but it will happen after first point. ... View more

Re: Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎01-27-2017 10:22 PM
‎01-27-2017 10:22 PM
@Kuldeep Kulkarni Many thanks. I had already seem them. Is there any cli rather then curl utility for deploying blueprints? ... View more

Re: Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎01-27-2017 07:28 PM
‎01-27-2017 07:28 PM
I`ve found only curl or some custom app for deploying configuration via blueprints. Is there any other way of deploying it? ... View more

Deploying multinode HDP with puppet

by Expert Contributor in Support Questions
‎01-27-2017 12:18 PM
‎01-27-2017 12:18 PM
Dear colleagues, I am trying to deploy multinode HDP with puppet, but have some troubles with services preconfiguration. Regarding ambari install and machines configuration everything is understandable: 1) Setup ssh, libraries, shares etc. using puppet. 2) Install ambari, also puppet - https://forge.puppetlabs.com/vzach/ambari However the problem appears when configuring services. For example, hbase, MR, ozzie, etc. For this I had not found any puppet libraries. I`ve seen that there is possible to manage a cluster via blueprints or ambari shell. Also had found that there is some limited support of bigtop. The bigtop seems to be more capable, but had not found any description of using it with ambari. Would be very grateful for some help this question. ... View more

Re: Unable to connect to apache phoenix remotely u...

by Expert Contributor in Support Questions
‎01-03-2017 08:49 PM
1 Kudo
‎01-03-2017 08:49 PM
1 Kudo
Solution found. When hbase is started in standalone(non-distributed) mode zookeeper takes hostname as hbase master and places it in /hbase/master. This value is returned to remote host when it tries to connect. If hostname is not resolved from remote host, then connection fails. ... View more

Re: Unable to connect to apache phoenix remotely u...

by Expert Contributor in Support Questions
‎12-27-2016 11:29 PM
‎12-27-2016 11:29 PM
Checked hostname resolve. Hostname resolves to host ip. ... View more

Re: Unable to connect to apache phoenix remotely u...

by Expert Contributor in Support Questions
‎12-25-2016 08:26 AM
‎12-25-2016 08:26 AM
Thanks for fast reply. Hbase ports are binded to host interface and can be reached via telnet from other machines in network. I`ve increased log verbosity to DEBUG of hbase and phoenix, but receive almost the same as before: 2016-12-25 09:19:08,001 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x159350a18720009, timeout of 40000ms exceeded 2016-12-25 09:19:08,002 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x159350a18720009 2016-12-25 09:19:13,549 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:5916 2016-12-25 09:19:13,558 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.2:5916 2016-12-25 09:19:13,562 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x159350f38570009 with negotiated timeout 40000 for client /192.168.56.2:5916 2016-12-25 09:19:31,985 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C37691%2C1482653082873.default.1482653085286 2016-12-25 09:19:31,985 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C45415%2C1482653262902..meta.1482653269511.meta 2016-12-25 09:19:31,986 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C37691%2C1482653082873..meta.1482653089699.meta 2016-12-25 09:19:31,986 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C44016%2C1482653169692..meta.1482653176455.meta 2016-12-25 09:19:31,986 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C45415%2C1482653262902.default.1482653265020 2016-12-25 09:19:31,986 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: ubuntu-dev%2C44016%2C1482653169692.default.1482653172062 2016-12-25 09:19:31,988 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C37691%2C1482653082873.default.1482653085286 from archive 2016-12-25 09:19:31,988 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C45415%2C1482653262902..meta.1482653269511.meta from archive 2016-12-25 09:19:31,989 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C37691%2C1482653082873..meta.1482653089699.meta from archive 2016-12-25 09:19:31,990 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C44016%2C1482653169692..meta.1482653176455.meta from archive 2016-12-25 09:19:31,991 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C45415%2C1482653262902.default.1482653265020 from archive 2016-12-25 09:19:31,997 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/oldWALs/ubuntu-dev%2C44016%2C1482653169692.default.1482653172062 from archive 2016-12-25 09:19:32,028 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/archive/data/hbase/meta/1588230740/info/c4846c81d545457da46dd0e5a648fba4 from archive 2016-12-25 09:19:32,029 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/archive/data/hbase/meta/1588230740/info/53cc0a8971704b799d887e79118d36dc from archive 2016-12-25 09:19:32,030 DEBUG [ubuntu-dev,45751,1482653908596_ChoreService_1] cleaner.CleanerChore: Removing: file:/opt/hbase/hbase-1.2.4/data/archive/data/hbase/meta/1588230740/info/e3e3cd0fa27b43aa8f68292dd94963d8 from archive ... View more

Unable to connect to apache phoenix remotely using...

by Expert Contributor in Support Questions
‎12-24-2016 07:42 AM
‎12-24-2016 07:42 AM
Dear community, I am experiencing problems when connecting to phoenix using squirrel client. The problem comes when I am connecting remotely. Local connection works good. Client throws exception: java.util.concurrent.TimeoutException at java.util.concurrent.FutureTask.get(FutureTask.java:201) at net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.awaitConnection(OpenConnectionCommand.java:132) at net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand.access$100(OpenConnectionCommand.java:45) at net.sourceforge.squirrel_sql.client.mainframe.action.OpenConnectionCommand$2.run(OpenConnectionCommand.java:115) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Zookeeper log: 2016-12-22 23:48:32,854 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:16551 2016-12-22 23:48:32,859 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.2:16551 2016-12-22 23:48:32,861 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x159287d6c210110 with negotiated timeout 60000 for client /192.168.56.2:16551 2016-12-22 23:50:19,131 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.168.56.2:16551 which had sessionid 0x159287d6c210110 2016-12-22 23:50:38,959 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:16560 2016-12-22 23:50:38,968 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.2:16560 2016-12-22 23:50:38,973 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x159287d6c210119 with negotiated timeout 60000 for client /192.168.56.2:16560 2016-12-22 23:55:52,641 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:16640 2016-12-22 23:55:53,683 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.168.56.2:16640 (no session established for client) 2016-12-22 23:59:37,436 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:16696 2016-12-22 23:59:37,438 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to renew session 0x159287d6c210119 at /192.168.56.2:16696 2016-12-22 23:59:37,440 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x159287d6c210119 with negotiated timeout 60000 for client /192.168.56.2:16696 2016-12-23 00:02:46,660 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:16768 2016-12-23 00:02:46,661 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.2:16768 2016-12-23 00:02:46,666 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x15928c2dfb30012 with negotiated timeout 60000 for client /192.168.56.2:16768 2016-12-23 00:04:57,929 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.168.56.2:16696 which had sessionid 0x159287d6c210119 2016-12-23 00:04:57,930 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection for client /192.168.56.2:16768 which had sessionid 0x15928c2dfb30012 2016-12-24 08:22:45,304 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket connection from /192.168.56.2:2933 2016-12-24 08:22:45,314 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting to establish new session at /192.168.56.2:2933 2016-12-24 08:22:45,318 INFO org.apache.zookeeper.server.ZooKeeperServer: Established session 0x15928c2dfb30086 with negotiated timeout 60000 for client /192.168.56.2:2933 I am using following connection string: jdbc:phoenix:192.168.56.102:2181:/hbase The port 2181 on 192.168.56.102 is accessible via telnet. ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎07-20-2018 02:22 PM
Community Statistics
Member Since ‎03-08-2016 01:52 PM
Last Visited ‎07-20-2018 02:22 PM
Posts 84
Kudos received 12