Reply
Cloudera Employee
Posts: 70
Registered: ‎10-14-2014

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

Also, to clarify: we're talking about Azure Database for PostgreSQL (https://azure.microsoft.com/en-us/services/postgresql/) and Azure Database for MySQL (https://azure.microsoft.com/en-us/services/mysql/), correct?
Expert Contributor
Posts: 92
Registered: ‎10-02-2017

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

That is correct.

Cloudera Employee
Posts: 70
Registered: ‎10-14-2014

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

One more clarification: in the config file, are you using explicit databases keyword (https://github.com/cloudera/director-scripts/blob/master/configs/azure.reference.conf#L710), or are you using databaseTemplates (https://github.com/cloudera/director-scripts/blob/master/configs/azure.reference.conf#L746)?
Highlighted
Expert Contributor
Posts: 92
Registered: ‎10-02-2017

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

We are using option for existing external databases which we launch and prepare prior to bootstrapping the cluster.  I believe in the cluster config spec it's referred to as:

 

"(Option 2) Optional configuration for existing external databases"

 

So, basically we launch an Azure Postgresql database and run DDL to setup CDH service databases, users, roles and refer to this DB in the cluster definition spec passed to Altus Director.  We do the same for AWS deployments, using RDS for Postgresql.   As I mentioned earlier in the thread, this was working fine in Azure with Altus Director 2.8.  

Expert Contributor
Posts: 92
Registered: ‎10-02-2017

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

Any luck with your testing?  Can we expect a patch soon?

 

 

Thanks

Cloudera Employee
Posts: 70
Registered: ‎10-14-2014

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

Hi! Sorry for the delay. Director 6.2.1 was released today with this (hopefully) patched. Do you mind testing and letting me know? Thank you!
Expert Contributor
Posts: 92
Registered: ‎10-02-2017

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

I will absolutely test and let you know.  I should have results to report within the next few days.

Cloudera Employee
Posts: 70
Registered: ‎10-14-2014

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

Hi dturner. Just wondering if you had had a chance to test this -- let me know!

Expert Contributor
Posts: 92
Registered: ‎10-02-2017

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

[ Edited ]

Hello Mike.  We've started up another round of Azure deployments so I finally got to test 6.2.1.   This is what I've found...

 

 

/etc/cloudera-scm-server/db.properties looks like it's getting generated correctly now:

 

com.cloudera.cmf.db.type=postgresql
com.cloudera.cmf.db.host=c8976732-postgres-81740268.postgres.database.azure.com:5432
com.cloudera.cmf.db.name=cdh_manager
com.cloudera.cmf.db.user=cdh_manager@c8976732-postgres-81740268
com.cloudera.cmf.db.password=********

Cluster bootstrap continues much further along, however during cluster first run we still see failure during creation of Hive Metastore db tables as it doesn't like the username format required by Postgres.  Perhaps the validation logic requires tweaking?

 

+ [[ create_metastore_tables == create_metastore_tables ]]
+ '[' 5 -ge 5 ']'
+ skip_if_tables_exist metastore_db_py.properties
+ '[' -z metastore_db_py.properties ']'
+ DB_INFO_FILENAME=metastore_db_py.properties
+ JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar
+ [[ -z '' ]]
+ JDBC_JARS_CLASSPATH='/usr/share/cmf/lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar'
++ /usr/java/default/bin/java -Djava.net.preferIPv4Stack=true -cp '/usr/share/cmf/lib/*:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar' com.cloudera.cmf.service.hive.HiveMetastoreDbUtil /run/cloudera-scm-agent/process/71-hive-metastore-create-tables/metastore_db_py.properties unused --printTableCount
Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: Name must contain only alphanumeric characters and underscores: 'cdh_hive@c8976732-postgres-81740268'
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.countTables(HiveMetastoreDbUtil.java:201)
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.printTableCount(HiveMetastoreDbUtil.java:281)
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.main(HiveMetastoreDbUtil.java:331)
Caused by: java.lang.IllegalArgumentException: Name must contain only alphanumeric characters and underscores: 'cdh_hive@c8976732-postgres-81740268'

 

It would appear to be complaining about the '@' in the username, however this is required by Postgres.  This point is proven if you remove the '@' from the username in Hive config and retry the command:

 

Exception in thread "main" java.lang.RuntimeException: org.postgresql.util.PSQLException: FATAL: Invalid Username specified. Please check the Username and retry connection. The Username should be in <username@hostname> format.
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.countTables(HiveMetastoreDbUtil.java:201)
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.printTableCount(HiveMetastoreDbUtil.java:281)
	at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.main(HiveMetastoreDbUtil.java:331)
Caused by: org.postgresql.util.PSQLException: FATAL: Invalid Username specified. Please check the Username and retry connection. The Username should be in <username@hostname> format.
Cloudera Employee
Posts: 70
Registered: ‎10-14-2014

Re: Altus Director 6.2 Azure deployment fails when using Azure Postgresql

Ooof. Looks like this is an issue in the Cloudera manager front. Let me ping the CM devs and they can take a look at this.