Ambari-server 2.7.3.0-139 setup doesn't work as expected

[#1]

The non-interactive setup of Ambari seems to be broken. It looks like the /etc/ambari-server/conf/ambari.properties file is not updated when using the ambari-server setup CLI flags (non-interactive setup).

Has anyone ever had the same issue?

Environment

• CentOS Linux release 7.6.1810 (Core)
• http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0 repo for ambari-server

Steps to reproduce

sudo yum install -y postgresql-jdbc ambari-server
sudo ambari-server setup --help

Using python /usr/bin/python
Setup ambari-server
Usage: ambari-server.py action [options]

Options:
-h, --help show this help message and exit
-v, --verbose Print verbose status messages
-s, --silent Silently accepts default prompt values. For db-cleanup
command, silent mode will stop ambari server.

Database options (command need to include all options):
--database=DBMS Database to use
embedded|oracle|mysql|mssql|postgres|sqlanywhere
--databasehost=DATABASE_HOST
Hostname of database server
--databaseport=DATABASE_PORT
Database port
--databasename=DATABASE_NAME
Database/Service name or ServiceID
--databaseusername=DATABASE_USERNAME
Database user login
--databasepassword=DATABASE_PASSWORD
Database user password

JDBC options (command need to include all options):
--jdbc-driver=JDBC_DRIVER
Specifies the path to the JDBC driver JAR file or
archive with all required files(jdbc jar, libraries
and etc), for the database type specified with the
--jdbc-db option. Used only with --jdbc-db option.
Archive is supported only for sqlanywhere database.
--jdbc-db=JDBC_DB Specifies the database type
[postgres|mysql|mssql|oracle|hsqldb|sqlanywhere] for
the JDBC driver specified with the --jdbc-driver
option. Used only with --jdbc-driver option.

Other options:
-j JAVA_HOME, --java-home=JAVA_HOME
Use specified java_home. Must be valid on all hosts
--stack-java-home=STACK_JAVA_HOME
Use specified java_home for stack services. Must be
valid on all hosts
--skip-view-extraction
Skip extraction of system views
--postgresschema=POSTGRES_SCHEMA
Postgres database schema name
--sqla-server-name=SQLA_SERVER_NAME
SQL Anywhere server name
--sidorsname=SID_OR_SNAME
Oracle database identifier type, Service ID/Service
Name sid|sname
--enable-lzo-under-gpl-license
Automatically accepts GPL license
--master-key=MASTER_KEY
Master key for encrypting passwords

sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --postgresschema=ambari --databaseusername=ambari --databasepassword=ambari --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar 

 sudo ambari-server start

Using python /usr/bin/python
Starting ambari-server
ERROR: Exiting with exit code -1.
REASON: DB Name property not set in config file.
- If this is a new setup, then run the "ambari-server setup" command to create the user
- If this is an upgrade of an existing setup, run the "ambari-server upgrade" command.
Refer to the Ambari documentation for more information on setup and upgrade.

The content of the /etc/ambari-server/conf/ambari.properties file AFTER running the ambari-server setup command is provided below:

#
# Copyright 2011 The Apache Software Foundation
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
#Mon Sep 09 19:26:55 UTC 2019
agent.package.install.task.timeout=1800
agent.stack.retry.on_repo_unavailability=false
agent.stack.retry.tries=5
agent.task.timeout=900
agent.threadpool.size.max=25
ambari.python.wrap=ambari-python-wrap
bootstrap.dir=/var/run/ambari-server/bootstrap
bootstrap.script=/usr/lib/ambari-server/lib/ambari_server/bootstrap.py
bootstrap.setup_agent.script=/usr/lib/ambari-server/lib/ambari_server/setupAgent.py
client.threadpool.size.max=25
common.services.path=/var/lib/ambari-server/resources/common-services
custom.action.definitions=/var/lib/ambari-server/resources/custom_action_definitions
custom.postgres.jdbc.name=postgresql-jdbc.jar
extensions.path=/var/lib/ambari-server/resources/extensions
http.cache-control=no-store
http.charset=utf-8
http.pragma=no-cache
http.strict-transport-security=max-age=31536000
http.x-content-type-options=nosniff
http.x-frame-options=DENY
http.x-xss-protection=1; mode=block
java.releases=jdk1.8
java.releases.ppc64le=
jce.download.supported=true
jdk.download.supported=true
jdk1.8.desc=Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
jdk1.8.dest-file=jdk-8u112-linux-x64.tar.gz
jdk1.8.home=/usr/jdk64/
jdk1.8.jcpol-file=jce_policy-8.zip
jdk1.8.jcpol-url=http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-8.zip
jdk1.8.re=(jdk.*)/jre
jdk1.8.url=http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz
kerberos.keytab.cache.dir=/var/lib/ambari-server/data/cache
kerberos.operation.verify.kdc.trust=true
metadata.path=/var/lib/ambari-server/resources/stacks
mpacks.staging.path=/var/lib/ambari-server/resources/mpacks
pid.dir=/var/run/ambari-server
previous.custom.postgres.jdbc.name=postgresql-jdbc.jar
recommendations.artifacts.lifetime=1w
recommendations.dir=/var/run/ambari-server/stack-recommendations
resources.dir=/var/lib/ambari-server/resources
rolling.upgrade.skip.packages.prefixes=
security.server.disabled.ciphers=TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384|TLS_RSA_WITH_AES_256_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384|TLS_DHE_RSA_WITH_AES_256_CBC_SHA256|TLS_DHE_DSS_WITH_AES_256_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA|TLS_RSA_WITH_AES_256_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA|TLS_ECDH_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_RSA_WITH_AES_256_CBC_SHA|TLS_DHE_DSS_WITH_AES_256_CBC_SHA|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256|TLS_RSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_RSA_WITH_AES_128_CBC_SHA256|TLS_DHE_DSS_WITH_AES_128_CBC_SHA256|TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA|TLS_RSA_WITH_AES_128_CBC_SHA|TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA|TLS_ECDH_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_RSA_WITH_AES_128_CBC_SHA|TLS_DHE_DSS_WITH_AES_128_CBC_SHA|TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA|TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA|SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA|TLS_EMPTY_RENEGOTIATION_INFO_SCSV|TLS_DH_anon_WITH_AES_256_CBC_SHA256|TLS_ECDH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_256_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA256|TLS_ECDH_anon_WITH_AES_128_CBC_SHA|TLS_DH_anon_WITH_AES_128_CBC_SHA|TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA|SSL_DH_anon_WITH_3DES_EDE_CBC_SHA|SSL_RSA_WITH_DES_CBC_SHA|SSL_DHE_RSA_WITH_DES_CBC_SHA|SSL_DHE_DSS_WITH_DES_CBC_SHA|SSL_DH_anon_WITH_DES_CBC_SHA|SSL_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA|SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA|SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA|TLS_RSA_WITH_NULL_SHA256|TLS_ECDHE_ECDSA_WITH_NULL_SHA|TLS_ECDHE_RSA_WITH_NULL_SHA|SSL_RSA_WITH_NULL_SHA|TLS_ECDH_ECDSA_WITH_NULL_SHA|TLS_ECDH_RSA_WITH_NULL_SHA|TLS_ECDH_anon_WITH_NULL_SHA|SSL_RSA_WITH_NULL_MD5|TLS_KRB5_WITH_3DES_EDE_CBC_SHA|TLS_KRB5_WITH_3DES_EDE_CBC_MD5|TLS_KRB5_WITH_DES_CBC_SHA|TLS_KRB5_WITH_DES_CBC_MD5|TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA|TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5
security.server.keys_dir=/var/lib/ambari-server/keys
server.connection.max.idle.millis=900000
server.execution.scheduler.isClustered=false
server.execution.scheduler.maxDbConnections=5
server.execution.scheduler.maxThreads=5
server.execution.scheduler.misfire.toleration.minutes=480
server.fqdn.service.url=http://169.254.169.254/latest/meta-data/public-hostname
server.http.session.inactive_timeout=1800
server.python.log.level=INFO
server.python.log.name=ambari-server-command.log
server.stages.parallel=true
server.task.timeout=1200
server.tmp.dir=/var/lib/ambari-server/data/tmp
server.version.file=/var/lib/ambari-server/resources/version
shared.resources.dir=/usr/lib/ambari-server/lib/ambari_commons/resources
skip.service.checks=false
stackadvisor.script=/var/lib/ambari-server/resources/scripts/stack_advisor.py
ulimit.open.files=65536
upgrade.parameter.convert.hive.tables.timeout=86400
upgrade.parameter.move.hive.tables.timeout=86400
user.inactivity.timeout.default=0
user.inactivity.timeout.role.readonly.default=0
views.ambari.request.connect.timeout.millis=30000
views.ambari.request.read.timeout.millis=45000
views.http.cache-control=no-store
views.http.charset=utf-8
views.http.pragma=no-cache
views.http.strict-transport-security=max-age=31536000
views.http.x-content-type-options=nosniff
views.http.x-frame-options=SAMEORIGIN
views.http.x-xss-protection=1; mode=block
views.request.connect.timeout.millis=5000
views.request.read.timeout.millis=10000
views.skip.home-directory-check.file-system.list=wasb,adls,adl
webapp.dir=/usr/lib/ambari-server/web

[#2]

@moritz 

By running the following command , you are telling Ambari to use an Existing Postgres DB instance in this case ambari will expect you to create the Db user and setup the DB schema on your own inside the PostgresDB (ambari will not do it for you).

# sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --postgresschema=ambari --databaseusername=ambari --databasepassword=ambari --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

Using existing Postgres DB instance for Ambari DB means we will need to make sure that After running the above "setup" command and before running the "start" command you already have performed the following task on your Postgres Database explicitly.

https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/administering-ambari/content/amb_using_ambari_...

Are you sure that before starting Ambari DB you had performed the instructions mentioned in the above link?

If you want Ambari to setup the PostgresDB (create DB user/ create various tables ..etc) for you then you may want to run the setup command without specifying the Postgres DB details so that ambari will think that it has to setup the Postgres DB for you in the Localhost with default settings like. default ambari DB username will be "ambari" default Postgres Db password will. be "bigdata" ...etc

# sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license -s

.

[#3]

you can give like this , if you want to use the embedded postgresql database of ambari .

sudo ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=embedded

reference : https://github.com/apache/ambari/blob/d44c1b118f8e17c1af83dbd71a06f1be2e82c4c3/ambari-server/src/mai...

https://github.com/apache/ambari/blob/d44c1b118f8e17c1af83dbd71a06f1be2e82c4c3/ambari-server/src/mai... 

[#4]

Thanks for your answers @akhilsnaik and @jsensharma.

I forgot to mention it but I am using a database created by following the documentation mentioned by jsensharma.

I already tried to use the embedded database by only using the --silent flag and it works, but in my case, I want a separate database. After installing it, I have created the ambari user, database and schema and I have loaded the /var/lib/ambari-server/resources/Ambari-DDL-Postgres-CREATE.sql script with no warning or error.

Thanks

Thomas

[#5 · Accepted solution]

@moritz

I see in your "ambari.properties" there are no JDBC settings appeared because looks like you are mixing the custom driver setup arguments along with the JDBC DB setup arguments in a single setup commands. Thats the reason the JDBC settings are not getting persisted.

Please try the following sequence of setup.

Step-1). Setup the JDBC DB settings.  After this step you should see that your ambari.properties file has the JDBC entries in it. 

# ambari-server setup --java-home /usr/lib/jvm/java-1.8.0-openjdk/jre --enable-lzo-under-gpl-license --database=postgres --databasehost=localhost --databaseport=5432 --databasename=ambari --databaseusername=ambari --postgresschema=ambari --databasepassword=ambari -s

Step-2). Now Setup the Custom Postgres JDBC Driver of your own choice using the following command. However, usually ambari ships a Postgres Driver here "/usr/lib/ambari-server/postgresql-*"

# ambari-server setup --jdbc-db=postgres --jdbc-driver=/usr/share/java/postgresql-jdbc.jar

Step-3). Now run the following command to verify the JDBC settings are reflected properly or not?

Example:

# grep 'jdbc' /etc/ambari-server/conf/ambari.properties
custom.postgres.jdbc.name=postgresql-jdbc.jar
server.jdbc.connection-pool=internal
server.jdbc.database=postgres
server.jdbc.database_name=ambari
server.jdbc.driver=org.postgresql.Driver
server.jdbc.hostname=localhost
server.jdbc.port=5432
server.jdbc.postgres.schema=ambari
server.jdbc.rca.driver=org.postgresql.Driver
server.jdbc.rca.url=jdbc:postgresql://localhost:5432/ambari
server.jdbc.rca.user.name=ambari
server.jdbc.rca.user.passwd=/etc/ambari-server/conf/password.dat
server.jdbc.url=jdbc:postgresql://localhost:5432/ambari
server.jdbc.user.name=ambari
server.jdbc.user.passwd=/etc/ambari-server/conf/password.dat

.

[#6]

Thanks a lot for your answer @jsensharma, using your command makes everything work as expected!

I didn't try this before because I was a bit confused about the impact of the --silent flag. I was assuming (wrongly) that using --silent implied to have all the parameters set to their default values.