eric_deleforter
Explorer

Re: Namenode heap usage has increased dramatically...

by Explorer in Support Questions
‎08-08-2019 02:56 PM
‎08-08-2019 02:56 PM
Hi Could you compare with the number of files on the same period ? It could be an explanation. Look the size of the files too, little size files and empty files could be an other explanation. Regards. ... View more

Re: Regarding to activate HIVE ACID transactions o...

by Explorer in Support Questions
‎08-08-2019 09:50 AM
‎08-08-2019 09:50 AM
Hi @Leonardo Araujo, don't know if you did it. About the collateral effect on 2.6.4, there is one to take in account, the Hive Acid activation change the transactional manager from : hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager To hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager The big impact is the way of Hive manage the locks even for no transactionnal tables as describe in Hive documentation about LockManager . This could dramatically increase processing times of your jobs using Hive. You could use the parameter hive.txn.strict.locking.mode to manage the behavior of the LockManager even if the documentation said that it is only available since Hive 2.2, use set -v to show presence of this parameter in Hive 1.2.1000 . You could hit 2 bugs too : HIVE-17900 : Compaction of ACID table might fail if table is partitioned by more than one column HIVE-18293 : Compaction of ACID table failed when using doas Hope this will help. Best regards, Eric. ... View more

Re: Hive metatstore won't start after Ambari Bluep...

by Explorer in Support Questions
‎10-24-2018 02:57 PM
1 Kudo
‎10-24-2018 02:57 PM
1 Kudo
Hello Clement and David Yes, the Failed to get database default, returning NoSuchObjectException is not a very clear message, this mean that the default Hive database on hdfs is not reachable : hdfs://clustername/user/hive/warehouse/ So this is not a postgresql problem but a hdfs client problem. Regards ... View more

Re: Hive metatstore won't start after Ambari Bluep...

by Explorer in Support Questions
‎10-23-2018 03:13 PM
‎10-23-2018 03:13 PM
Hi David Did you solve this problem ? Best regards. Eric. ... View more

Re: Version of PostgreSQL for Ambari 2.6.2.2 + HDP...

by Explorer in Support Questions
‎08-06-2018 07:08 AM
‎08-06-2018 07:08 AM
Hi Always use the newer version, less bugs, more security and functionality. Even if it is not in the matrix, I use postgresl 9.5 for ambari, hive, ranger, oozie, hue. Keep in mind that if you plan to upgrade to HDP 3 and ambari 2.7 you have to upgrade to postgresql 10.2. Regards. ... View more

Re: Registered fro HDPCA Exam, where can i get Moc...

by Explorer in Support Questions
‎08-01-2018 04:43 PM
1 Kudo
‎08-01-2018 04:43 PM
1 Kudo
Hi You can practice the exam on AWS : https://fr.hortonworks.com/services/training/certification/hdpca-certification/ you have to follow this doc for using the AWS practice : https://2xbbhjxc6wk3v21p62t8n4d4-wpengine.netdna-ssl.com/wp-content/uploads/2015/04/HDPCA-PracticeExamGuide.pdf Just be carefull the AMI are not available on all the AWS zone, try us-east-1 (USA East, Virginie). Good luck. Regards. Eric. ... View more

Re: Hue port redirecting

by Explorer in Support Questions
‎08-01-2018 04:16 PM
‎08-01-2018 04:16 PM
Hi Have you check that Hue is listening on the right port 443 (https) ? 443 is the https default port, so your browser remove it from the url and replace it by https:// in front of your ip address. You could try with another port if you want to verify, ex: 15001 Regards ... View more

Re: How much latency can the Ambari database have?

by Explorer in Support Questions
‎08-01-2018 02:54 PM
‎08-01-2018 02:54 PM
Hi First what is your database type ? In Ambari alerts you have a "Ambari Server Performance" alert with default treshold : warning : 3 000 ms critical : 5 000 ms So 8 ms should not issue an alert. What do you mean by latency ? The time to open a connection to the database ? The time for a simple request to answer ? On my Postgresql database I have many queries answers under 1 ms and others takes 3 ms to 9 ms, but I have other db that ambari ( hive, oozie, hue, ranger, ...) Regards. Eric ... View more

Re: How to setup Ambari with an external Postgresq...

by Explorer in Support Questions
‎08-01-2018 02:26 PM
13 Kudos
‎08-01-2018 02:26 PM
13 Kudos
Hi Here is the answer : After setup your postgresql external database access with ambari-server setup, you have to edit the file /etc/ambari-server/conf/ambari.properties and modify the server.jdbc.url parameter to add the activation of the ssl like this : server.jdbc.url=jdbc:postgresql://<HOSTNAME>:<PORT>/ambari?ssl=true Once done, you are able to start the ambari installation with a SSL connection to your external Postgresql database. For more information on all the options you could add on the connect string, you have the jdbc.postgresql documentation here : https://jdbc.postgresql.org/documentation/head/connect.html#ssl Eric ... View more

How to setup Ambari with an external Postgresql db...

by Explorer in Support Questions
‎08-01-2018 02:20 PM
4 Kudos
‎08-01-2018 02:20 PM
4 Kudos
When installing a cluster with a very secure environement you could want to setup an external Postgresql cluster database with HA and a connection pooler like pgbouncer for reducing the impact of opening new sessions on Postgresql. All the connections should be with SSL/TLS : Client -> Pgbouncer and Pgbouncer -> Postgresql The problem was that configuring Ambari with the ambari-server setup don't give you the oportunity to setup SSL connection and ambari is not able to connect to the database. You will find this error in the logs : 26 Jul 2018 18:56:39,202 ERROR [main] DBAccessorImpl:119 - Error while creating database accessor org.postgresql.util.PSQLException: ERROR: SSL required at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:400) at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:173) at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64) at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:138) at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29) at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21) at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31) at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) at org.postgresql.Driver.makeConnection(Driver.java:410) at org.postgresql.Driver.connect(Driver.java:280) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.apache.ambari.server.orm.DBAccessorImpl.<init>(DBAccessorImpl.java:93) at org.apache.ambari.server.orm.DBAccessorImpl$$FastClassByGuice$$86dbc63e.newInstance(<generated>) at com.google.inject.internal.cglib.reflect.$FastConstructor.newInstance(FastConstructor.java:40) at com.google.inject.internal.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:60) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:65) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:54) at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53) at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:65) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53) at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:110) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:94) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:254) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1031) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:65) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:40) at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:53) at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:116) at com.google.inject.internal.InjectionRequestProcessor$StaticInjection$1.call(InjectionRequestProcessor.java:110) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1024) at com.google.inject.internal.InjectionRequestProcessor$StaticInjection.injectMembers(InjectionRequestProcessor.java:110) at com.google.inject.internal.InjectionRequestProcessor.injectMembers(InjectionRequestProcessor.java:78) at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:170) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109) at com.google.inject.Guice.createInjector(Guice.java:95) at com.google.inject.Guice.createInjector(Guice.java:72) at com.google.inject.Guice.createInjector(Guice.java:62) at org.apache.ambari.server.controller.AmbariServer.main(AmbariServer.java:1045) So, the question is how to setup your ambari server for connecting to the database without desactivating SSL mode ? Eric. ... View more
Labels:

Re: Ambari with Postgres HA

by Explorer in Community Articles
‎08-01-2018 08:13 AM
1 Kudo
‎08-01-2018 08:13 AM
1 Kudo
If you want a Postgresql HA for your Ambari and the other components like Ranger, Hive, Oozie, you have to use an external postgresql database, not the embeded. In front of your Postgresql HA, use a connection pooler like pgbouncer or pgpool, so you will reduce the impact of opening a client connection to the database by keeping sessions open between the connection pooler and the database. ... View more
Contact Me
Online
Offline
Last Visited
‎05-26-2020 04:20 PM
Public Statistics
Date Registered ‎05-05-2017 09:10 AM
Last Visited ‎05-26-2020 04:20 PM
Total Messages Posted 11
Total Kudos Received 20