Ambari-server 2.7.3.0-139 setup doesn't work as expected

Question

The non-interactive setup of Ambari seems to be broken. It looks like the /etc/ambari-server/conf/ambari.properties file is not updated when using the ambari-server setup CLI flags (non-interactive setup).

Has anyone ever had the same issue?

Environment

CentOS Linux release 7.6.1810 (Core)
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0 repo for ambari-server

Steps to reproduce

sudo yum install -y postgresql-jdbc ambari-server sudo ambari-server setup --help

Using python /usr/bin/python Setup ambari-server Usage: ambari-server.py action [options] Options: -h, --help show this help message and exit -v, --verbose Print verbose status messages -s, --silent Silently accepts default prompt values. For db-cleanup command, silent mode will stop ambari server. Database options (command need to include all options): --database=DBMS Database to use embedded|oracle|mysql|mssql|postgres|sqlanywhere --databasehost=DATABASE_HOST Hostname of database server --databaseport=DATABASE_PORT Database port --databasename=DATABASE_NAME Database/Service name or ServiceID --databaseusername=DATABASE_USERNAME Database user login --databasepassword=DATABASE_PASSWORD Database user password JDBC options (command need to include all options): --jdbc-driver=JDBC_DRIVER Specifies the path to the JDBC driver JAR file or archive with all required files(jdbc jar, libraries and etc), for the database type specified with the --jdbc-db option. Used only with --jdbc-db option. Archive is supported only for sqlanywhere database. --jdbc-db=JDBC_DB Specifies the database type [postgres|mysql|mssql|oracle|hsqldb|sqlanywhere] for the JDBC driver specified with the --jdbc-driver option. Used only with --jdbc-driver option. Other options: -j JAVA_HOME, --java-home=JAVA_HOME Use specified java_home. Must be valid on all hosts --stack-java-home=STACK_JAVA_HOME Use specified java_home for stack services. Must be valid on all hosts --skip-view-extraction Skip extraction of system views --postgresschema=POSTGRES_SCHEMA Postgres database schema name --sqla-server-name=SQLA_SERVER_NAME SQL Anywhere server name --sidorsname=SID_OR_SNAME Oracle database identifier type, Service ID/Service Name sid|sname --enable-lzo-under-gpl-license Automatically accepts GPL license --master-key=MASTER_KEY Master key for encrypting passwords

sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --postgresschema=ambari --databaseusername=ambari --databasepassword=ambari --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

sudo ambari-server start

Using python /usr/bin/python Starting ambari-server ERROR: Exiting with exit code -1. REASON: DB Name property not set in config file. - If this is a new setup, then run the "ambari-server setup" command to create the user - If this is an upgrade of an existing setup, run the "ambari-server upgrade" command. Refer to the Ambari documentation for more information on setup and upgrade.

The content of the /etc/ambari-server/conf/ambari.properties file AFTER running the ambari-server setup command is provided below:

# # Copyright 2011 The Apache Software Foundation # # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # #Mon Sep 09 19:26:55 UTC 2019 agent.package.install.task.timeout=1800 agent.stack.retry.on_repo_unavailability=false agent.stack.retry.tries=5 agent.task.timeout=900 agent.threadpool.size.max=25 ambari.python.wrap=ambari-python-wrap bootstrap.dir=/var/run/ambari-server/bootstrap bootstrap.script=/usr/lib/ambari-server/lib/ambari_server/bootstrap.py bootstrap.setup_agent.script=/usr/lib/ambari-server/lib/ambari_server/setupAgent.py client.threadpool.size.max=25 common.services.path=/var/lib/ambari-server/resources/common-services custom.action.definitions=/var/lib/ambari-server/resources/custom_action_definitions custom.postgres.jdbc.name=postgresql-jdbc.jar extensions.path=/var/lib/ambari-server/resources/extensions http.cache-control=no-store http.charset=utf-8 http.pragma=no-cache http.strict-transport-security=max-age=31536000 http.x-content-type-options=nosniff http.x-frame-options=DENY http.x-xss-protection=1; mode=block java.releases=jdk1.8 java.releases.ppc64le= jce.download.supported=true jdk.download.supported=true jdk1.8.desc=Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 jdk1.8.dest-file=jdk-8u112-linux-x64.tar.gz jdk1.8.home=/usr/jdk64/ jdk1.8.jcpol-file=jce_policy-8.zip jdk1.8.jcpol-url=http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip jdk1.8.re=(jdk.*)/jre jdk1.8.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz kerberos.keytab.cache.dir=/var/lib/ambari-server/data/cache kerberos.operation.verify.kdc.trust=true metadata.path=/var/lib/ambari-server/resources/stacks mpacks.staging.path=/var/lib/ambari-server/resources/mpacks pid.dir=/var/run/ambari-server previous.custom.postgres.jdbc.name=postgresql-jdbc.jar recommendations.artifacts.lifetime=1w recommendations.dir=/var/run/ambari-server/stack-recommendations resources.dir=/var/lib/ambari-server/resources rolling.upgrade.skip.packages.prefixes= security.server.disabled.ciphers=TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384|TLS_RSA_WITH_AES_256_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384|TLS_DHE_RSA_WITH_AES_256_CBC_SHA256|TLS_DHE_DSS_WITH_AES_256_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA|TLS_RSA_WITH_AES_256_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_DSS_WITH_AES_256_CBC_SHA|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256|TLS_RSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_DSS_WITH_AES_128_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA|TLS_RSA_WITH_AES_128_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_DSS_WITH_AES_128_CBC_SHA|TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA|TLS_EMPTY_RENEGOTIATION_INFO_SCSV|TLS_DH_anon_WITH_AES_256_CBC_SHA256|TLS_ECDH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA256|TLS_ECDH_anon_WITH_AES_128_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA|TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA|SSL_DH_anon_WITH_3DES_EDE_CBC_SHA|SSL_RSA_WITH_DES_CBC_SHA|SSL_DHE_RSA_WITH_DES_CBC_SHA|SSL_DHE_DSS_WITH_DES_CBC_SHA|SSL_DH_anon_WITH_DES_CBC_SHA|SSL_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA|SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA|TLS_RSA_WITH_NULL_SHA256|TLS_ECDHE_ECDSA_WITH_NULL_SHA|TLS_ECDHE_RSA_WITH_NULL_SHA|SSL_RSA_WITH_NULL_SHA|TLS_ECDH_ECDSA_WITH_NULL_SHA|TLS_ECDH_RSA_WITH_NULL_SHA|TLS_ECDH_anon_WITH_NULL_SHA|SSL_RSA_WITH_NULL_MD5|TLS_KRB5_WITH_3DES_EDE_CBC_SHA|TLS_KRB5_WITH_3DES_EDE_CBC_MD5|TLS_KRB5_WITH_DES_CBC_SHA|TLS_KRB5_WITH_DES_CBC_MD5|TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA|TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5 security.server.keys_dir=/var/lib/ambari-server/keys server.connection.max.idle.millis=900000 server.execution.scheduler.isClustered=false server.execution.scheduler.maxDbConnections=5 server.execution.scheduler.maxThreads=5 server.execution.scheduler.misfire.toleration.minutes=480 server.fqdn.service.url=http://169.254.169.254/latest/meta-data/public-hostname server.http.session.inactive_timeout=1800 server.python.log.level=INFO server.python.log.name=ambari-server-command.log server.stages.parallel=true server.task.timeout=1200 server.tmp.dir=/var/lib/ambari-server/data/tmp server.version.file=/var/lib/ambari-server/resources/version shared.resources.dir=/usr/lib/ambari-server/lib/ambari_commons/resources skip.service.checks=false stackadvisor.script=/var/lib/ambari-server/resources/scripts/stack_advisor.py ulimit.open.files=65536 upgrade.parameter.convert.hive.tables.timeout=86400 upgrade.parameter.move.hive.tables.timeout=86400 user.inactivity.timeout.default=0 user.inactivity.timeout.role.readonly.default=0 views.ambari.request.connect.timeout.millis=30000 views.ambari.request.read.timeout.millis=45000 views.http.cache-control=no-store views.http.charset=utf-8 views.http.pragma=no-cache views.http.strict-transport-security=max-age=31536000 views.http.x-content-type-options=nosniff views.http.x-frame-options=SAMEORIGIN views.http.x-xss-protection=1; mode=block views.request.connect.timeout.millis=5000 views.request.read.timeout.millis=10000 views.skip.home-directory-check.file-system.list=wasb,adls,adl webapp.dir=/usr/lib/ambari-server/web

jsensharma · Accepted Answer

@moritz

I see in your "ambari.properties" there are no JDBC settings appeared because looks like you are mixing the custom driver setup arguments along with the JDBC DB setup arguments in a single setup commands. Thats the reason the JDBC settings are not getting persisted.

Please try the following sequence of setup.

Step-1). Setup the JDBC DB settings. After this step you should see that your ambari.properties file has the JDBC entries in it.

# ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --databaseusername=ambari --postgresschema=ambari --databasepassword=ambari -s

Step-2). Now Setup the Custom Postgres JDBC Driver of your own choice using the following command. However, usually ambari ships a Postgres Driver here "/usr/lib/ambari-server/postgresql-*"

# ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

Step-3). Now run the following command to verify the JDBC settings are reflected properly or not?

Example:

# grep 'jdbc' /etc/ambari-server/conf/ambari.properties
custom.postgres.jdbc.name=postgresql-jdbc.jar
server.jdbc.connection-pool=internal
server.jdbc.database=postgres
server.jdbc.database_name=ambari
server.jdbc.driver=org.postgresql.Driver
server.jdbc.hostname=localhost
server.jdbc.port=5432
server.jdbc.postgres.schema=ambari
server.jdbc.rca.driver=org.postgresql.Driver
server.jdbc.rca.url=jdbc:postgresql://localhost:5432/ambari
server.jdbc.rca.user.name=ambari
server.jdbc.rca.user.passwd=/etc/ambari-server/conf/password.dat
server.jdbc.url=jdbc:postgresql://localhost:5432/ambari
server.jdbc.user.name=ambari
server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat

.

jsensharma · Answer

@moritz

By running the following command , you are telling Ambari to use an Existing Postgres DB instance in this case ambari will expect you to create the Db user and setup the DB schema on your own inside the PostgresDB (ambari will not do it for you).

# sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --postgresschema=ambari --databaseusername=ambari --databasepassword=ambari --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

Using existing Postgres DB instance for Ambari DB means we will need to make sure that After running the above "setup" command and before running the "start" command you already have performed the following task on your Postgres Database explicitly.

https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/administering-ambari/content/amb_using_ambari_with_postgresql.html

Are you sure that before starting Ambari DB you had performed the instructions mentioned in the above link?

If you want Ambari to setup the PostgresDB (create DB user/ create various tables ..etc) for you then you may want to run the setup command without specifying the Postgres DB details so that ambari will think that it has to setup the Postgres DB for you in the Localhost with default settings like. default ambari DB username will be "ambari" default Postgres Db password will. be "bigdata" ...etc

# sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license -s

.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.