CDH 5.1.3.1 installation issues - Sqoop2

[#1]

After upgrading from 5.1.2.1 to 5.1.3.1. the cluster fails to start the Sqoop2 server.

It fails to load ConnectionManagers. The only thing I changed in sqoop was that I added the Microsoft JDBC driver (jar file) to the lib directory. Even after removing the file the cluster fails to start the Sqoop2 server.

[#2]

Please provide the exact error messages so we can assist you. You can paste
the log file to pastebin and provide the URL here if you wish

[#3 · Accepted solution]

I have googled the solution.

After setting the propperty org.apache.sqoop.connector.autoupgrade=true the Sqoop server started ok.

I dont know what this propperty means, but it worked 🙂

Thanks,

Tomas.

http://www.developerscloset.com/?p=961#respond

Sqoop Server Startup Failure: Upgrade required but not allowed
After an upgrade from CDH 5.0.2 to CDH 5.0.3, Sqoop failed to start with the following error: Server startup failure, Connector registration failed, Upgrade required but not allowed - Connector: generic-jdbc-connector.

To resolve this problem I had to add the following property to the Sqoop 2 Server Advanced Configuration Snippet (Safety Valve) for sqoop.properties. You can find this property under Cloudera Manager, Sqoop Service, Configuration, Sqoop 2 Server Default Group, and Advanced:

org.apache.sqoop.connector.autoupgrade=true

After the upgrade has completed successfully, the property can be removed.

[#4]

I encountered the same issue when upgrading from CDH 5.2.0 => 5.2.1

I did not find this support thread until after removing the sqoop2 service and reinstalling it. 

part of my logs:

Caused by: org.apache.sqoop.common.SqoopException: CONN_0007:Connector registration failed

Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0026:Upgrade required but not allowed - Connector: generic-jdbc-connector

The solution of using 

Cloudera Manager -> Sqoop2 -> Configuration -> Sqoop 2 Server Default Group -> Advanced

  -> Sqoop 2 Server Advanced Configuration Snippet (Safety Valve)

    org.apache.sqoop.connector.autoupgrade=true

worked for me. 

The Sqoop2 service was able to successfully start

Michael

[#5]

Normally you just run the Sqoop2 upgrade command whenever you upgrade Sqoop binaries.

[#6]

> Normally you just run the Sqoop2 upgrade command whenever you upgrade Sqoop binaries.

I don't understand. 

I did not explictly/exclusively upgrade the Sqoop binaries. Nor would I know how/where to "run the Sqoop2 upgrade command"

I upgraded the cluster from CDH 5.2.0 => CDH 5.2.1 through the Cloudera Manager web gui. 

The upgrade process restarted the cluster

the Sqoop2 service (and the Hue service which is dependent upon sqoop2) did not restart. 

Please advise. 

[#7]

Upgrading CDH will upgrade your Sqoop2 binaries since Sqoop2 is part of CDH.

Every time you upgrade Sqoop2 binaries you need to run the Sqoop2 upgrade command. Click on your Sqoop2 service and look for the upgrade command in the actions dropdown menu on the right. You have to stop the service first.

[#8]

> Upgrading CDH will upgrade your Sqoop2 binaries since Sqoop2 is part of CDH. 

OK

> Every time you upgrade Sqoop2 binaries you need to run the Sqoop2 upgrade command.

> Click on your Sqoop2 service and look for the upgrade command in the actions dropdown menu on the right.

> You have to stop the service first. 

Then, I am a little puzzled/confused. 

I understand your statements to imply that the Cloudera Manager upgrade wizard is broken and does not function properly with respect to Sqoop2.

> Every time you upgrade Sqoop2 binaries you need to run the Sqoop2 upgrade command. 

So, the cloudera manager upgrade wizard does not run the Sqoop2 upgrade command ... even though this is a required step. 

> You have to stop the service first. 

The Cloudera Manager upgrade wizard stops & then starts all of the services in the cluster. The Sqoop2 service fails to start and the cluster is left in a bad state. 

This is the behavior that I observed. 

Q: Are you saying that this is a known problem with the Cloudera Manager upgrade wizard? 

[#9]

Cloudera Manager's upgrade wizard currently only supports major upgrades, which are generally much more complicated. Since you upgraded from 5.2.0 to 5.2.1, the wizard wasn't applicable. For minor / maintenance upgrades, you need to follow the documentation. Stay tuned though, as we are making improvements here in a future release.

For major upgrades, the existing CM upgrade wizard will run the sqoop upgrade command for you.

I think that you clicked activate parcel, it prompted you to restart, and you thought that was an upgrade wizard when in fact it was just a simple cluster restart. We're making improvements to this specific experience as well.

[#10]

> I think that you clicked activate parcel, it prompted you to restart, and you thought that was an upgrade wizard

> when in fact it was just a simple cluster restart.

agreed. 

> We're making improvements to this specific experience as well.

Understood.

Thank you for the explanation.