Member since
05-05-2017
11
Posts
20
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1667 | 08-01-2018 04:43 PM | |
| 3813 | 08-01-2018 02:26 PM |
08-08-2019
09:50 AM
Hi @Leonardo Araujo, don't know if you did it. About the collateral effect on 2.6.4, there is one to take in account, the Hive Acid activation change the transactional manager from : hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager To hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager The big impact is the way of Hive manage the locks even for no transactionnal tables as describe in Hive documentation about LockManager . This could dramatically increase processing times of your jobs using Hive. You could use the parameter hive.txn.strict.locking.mode to manage the behavior of the LockManager even if the documentation said that it is only available since Hive 2.2, use set -v to show presence of this parameter in Hive 1.2.1000 . You could hit 2 bugs too : HIVE-17900 : Compaction of ACID table might fail if table is partitioned by more than one column HIVE-18293 : Compaction of ACID table failed when using doas Hope this will help. Best regards, Eric.
... View more
08-06-2018
07:08 AM
Hi Always use the newer version, less bugs, more security and functionality. Even if it is not in the matrix, I use postgresl 9.5 for ambari, hive, ranger, oozie, hue. Keep in mind that if you plan to upgrade to HDP 3 and ambari 2.7 you have to upgrade to postgresql 10.2. Regards.
... View more
08-01-2018
04:43 PM
1 Kudo
Hi You can practice the exam on AWS : https://fr.hortonworks.com/services/training/certification/hdpca-certification/ you have to follow this doc for using the AWS practice : https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2015/04/HDPCA-PracticeExamGuide.pdf Just be carefull the AMI are not available on all the AWS zone, try us-east-1 (USA East, Virginie). Good luck. Regards. Eric.
... View more
08-01-2018
02:26 PM
13 Kudos
Hi Here is the answer : After setup your postgresql external database access with ambari-server setup, you have to edit the file /etc/ambari-server/conf/ambari.properties and modify the server.jdbc.url parameter to add the activation of the ssl like this : server.jdbc.url=jdbc:postgresql://<HOSTNAME>:<PORT>/ambari?ssl=true Once done, you are able to start the ambari installation with a SSL connection to your external Postgresql database. For more information on all the options you could add on the connect string, you have the jdbc.postgresql documentation here : https://jdbc.postgresql.org/documentation/head/connect.html#ssl Eric
... View more
08-01-2018
02:20 PM
4 Kudos
When installing a cluster with a very secure environement you could want to setup an external Postgresql cluster database with HA and a connection pooler like pgbouncer for reducing the impact of opening new sessions on Postgresql. All the connections should be with SSL/TLS : Client -> Pgbouncer and Pgbouncer -> Postgresql The problem was that configuring Ambari with the ambari-server setup don't give you the oportunity to setup SSL connection and ambari is not able to connect to the database. You will find this error in the logs : 26 Jul 2018 18:56:39,202 ERROR [main] DBAccessorImpl:119 - Error while creating database accessor
org.postgresql.util.PSQLException: ERROR: SSL required
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:400)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:138)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:410)
at org.postgresql.Driver.connect(Driver.java:280)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.ambari.server.orm.DBAccessorImpl.<init>(DBAccessorImpl.java:93)
at org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance(<generated>)
at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40)
at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.Scopes$1$1.get(Scopes.java:65)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54)
at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.Scopes$1$1.get(Scopes.java:65)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254)
at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031)
at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
at com.google.inject.Scopes$1$1.get(Scopes.java:65)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40)
at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53)
at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:116)
at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:110)
at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024)
at com.google.inject.internal.InjectionRequestProcessor$StaticInjection.injectMembers(InjectionRequestProcessor.java:110)
at com.google.inject.internal.InjectionRequestProcessor.injectMembers(InjectionRequestProcessor.java:78)
at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:170)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
at com.google.inject.Guice.createInjector(Guice.java:95)
at com.google.inject.Guice.createInjector(Guice.java:72)
at com.google.inject.Guice.createInjector(Guice.java:62)
at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1045)
So, the question is how to setup your ambari server for connecting to the database without desactivating SSL mode ? Eric.
... View more
Labels:
- Labels:
-
Apache Ambari
08-01-2018
08:13 AM
1 Kudo
If you want a Postgresql HA for your Ambari and the other components like Ranger, Hive, Oozie, you have to use an external postgresql database, not the embeded. In front of your Postgresql HA, use a connection pooler like pgbouncer or pgpool, so you will reduce the impact of opening a client connection to the database by keeping sessions open between the connection pooler and the database.
... View more