Error while generating migration script for Ambari Infra Solr - Upgrading to HDP-3.0

[#1]

While upgrading the cluster from HDP-2.6 to HDP-3.0, not able to generate ambari-infra-solr migration script due to below error .

[root@hostname ambari-infra-solr-client]# /usr/bin/python /usr/lib/ambari-infra-solr-client/migrationConfigGenerator.py --ini-file $CONFIG_INI_LOCATION --host <ambari server> --port 8443 -s --cluster DEV --username rchaman --password rchaman --backup-base-path=/tmp/back/ --java-home /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre --ranger-hdfs-base-path=/ranger/audit
Start generating config file: ambari_solr_migration.ini ...
Get Ambari cluster details ...
Set JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre
Service detected: ZOOKEEPER
Zookeeper connection string: zk1.openstack:2181,zk2.openstack:2181,zk3.openstack:2181
Service detected: AMBARI_INFRA_SOLR
Infra Solr znode: /infra-solr_test
2018-11-14 14:38:59,104 - Get clusterstate.json: waiting for 5 seconds before retyring again (retry count: 1)
2018-11-14 14:39:04,169 - Get clusterstate.json: waiting for 5 seconds before retyring again (retry count: 2)

Solution - This solution will work if you have below error in solr.log file. This issue occurs when the infra-solr is restarted manually.Basically the ambari infra service should not be stopped unless done by ambari itself.. When Ambari uploads the latest version of security.json to zookeeper, it will be incompatible with the old version of Infra-Solr. Both the version use different classes. For example (look for authorization)

Before upgrade

{
  "authentication": {
    "class": "org.apache.solr.security.KerberosPlugin"
  },
  "authorization": {
    "class": "org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin",

After Upgrade

{
  "authentication": {
    "class": "org.apache.solr.security.KerberosPlugin"
  },
  "authorization": {
    "class": "org.apache.solr.security.InfraRuleBasedAuthorizationPlugin",

Due to this mismatch you might see this error in solr.log

(class not found: org.apache.solr.security.InfraRuleBasedAuthorizationPlugin )

Timestamp [main] WARN  [   ] org.apache.solr.core.CoreContainer (CoreContainer.java:401) - Couldn't add files from /opt/ambari_infra_solr/data
/lib to classpath: /opt/ambari_infra_solr/data/lib
Timestamp [main] ERROR [   ] org.apache.solr.servlet.SolrDispatchFilter (SolrDispatchFilter.java:141) - Could not start Solr. Check solr/home
property and the logs
Timestamp [main] ERROR [   ] org.apache.solr.common.SolrException (SolrException.java:159) - null:org.apache.solr.common.SolrException: Error
loading class 'org.apache.solr.security.InfraRuleBasedAuthorizationPlugin'
        at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:558)
        at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:627)
        at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:592)
        at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:585)
        at org.apache.solr.core.CoreContainer.initializeAuthorizationPlugin(CoreContainer.java:245)
        at org.apache.solr.core.CoreContainer.load(CoreContainer.java:420)
        at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:158)

Step1: Locate security.json file on node running Ambari infra instance and out of these on two below mentioned files, replace the new class with old class

/etc/ambari-infra-solr/conf/custom-security.json /var/lib/ambari-agent/cache/common-services/AMBARI_INFRA_SOLR/0.1.0/package/templates/infra-solr-security.json.j2

Replace org.apache.solr.security.InfraRuleBasedAuthorizationPlugin with org.apache.ambari.infra.security.InfraRuleBasedAuthorizationPlugin

Restart Ambari-infra and try to rerun the command

[root@rchaman ~]# export CONFIG_INI_LOCATION=ambari_solr_migration.ini
[root@rchaman ~]# /usr/bin/python /usr/lib/ambari-infra-solr-client/migrationConfigGenerator.py --ini-file $CONFIG_INI_LOCATION --host ambari-server --port 8443 -s --cluster DEV --username rchaman --password password --backup-base-path=/tmp/back/ --java-home /opt/jdk1.8.0_112/jre --ranger-hdfs-base-path=/ranger/audit
Start generating config file: ambari_solr_migration.ini ...
Get Ambari cluster details ...
Set JAVA_HOME: /opt/jdk1.8.0_112/jre
Service detected: ZOOKEEPER
Zookeeper connection string: zk1.openstack:2181,zk2.openstack:2181,zk3.openstack:2181
Service detected: AMBARI_INFRA_SOLR
Infra Solr znode: /infra-solr
Service detected: RANGER
Ranger Solr collection: ranger_audits
Ranger backup path: /tmp/back/ranger
Service detected: ATLAS
Atlas Solr collections: fulltext_index, edge_index, vertex_index
Atlas backup path: /tmp/back/atlas
Kerberos: enabled
Config file generation has finished successfully

Important - It is NOT RECOMMENDED to restart Ambari-infra during the upgrade process. Restart ambari-infra only when you get an issue and restart is unavoidable.

Once done resume with the steps mentioned in this link