question Re: CM update Sqoop Service Environment Safety Valve i in Support Questions

CM update Sqoop Service Environment Safety Valve i

praveenhm — Fri, 16 Sep 2022 08:56:56 GMT

Problem desc:

I need to update the 2 properties in sqoop, but I am not able do it through sqoop.propeties files as CM will overide it, Its need to be done in the Sqoop Service Environment Safety Valve in CM,

But when I did it in the sqoop2 service, in the advanced configuration,to setup key/value pair for "Sqoop Service Environment Safety Valve", below is the format.

org.apache.sqoop.connector.autoupgrade=true

org.apache.sqoop.framework.autoupgrade=true

I get the below error,

Could not parse: Sqoop Service Environment Safety Valve : Could not parse parameter 'sqoop_env_safety_valve'. Was expecting: valid variable name. Input: connector.autoupgrade=true

If i just give it like this it works

autoupgrade=true

but I need the whole string. I need help, what is format?

Thanks

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Wed, 09 Apr 2014 22:44:23 GMT

Hi,

Why do you want to enable auto upgrade? Sqoop has upgrade commands that will do this for you. Auto upgrade may have other side effects that could cause problems down the line.

If you really want to make this change, you need to modify the safety valve for sqoop.properties, not the environment safety valve.

Thanks,

Darren

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Wed, 09 Apr 2014 23:08:59 GMT

Thanks dlo for reply,

I started doing this because when I start the sqoop service in CM, I get the below error. So some one on the forum suggested that turning to true will help that,

Connector config urls: [jar:file:/hdp2014poc/cloudera/parcels/CDH-4.6.0-1.cdh4.6.0.p0.26/lib/sqoop2/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.2-cdh4.6.0.jar!/sqoopconnector.properties]

Connector [org.apache.sqoop.connector.jdbc.GenericJdbcConnector] initialized.

Marking transaction for rollback

Attempting transaction roll-back

Server startup failure
org.apache.sqoop.common.SqoopException: CONN_0007:Connector registration failed
	at org.apache.sqoop.connector.ConnectorManager.registerConnectors(ConnectorManager.java:234)
	at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:195)
	at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:51)
	at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:36)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206)
	at org.apache.catalina.core.StandardContext.start(StandardContext.java:4705)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
	at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
	at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
	at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
	at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
	at org.apache.catalina.core.StandardService.start(StandardService.java:525)
	at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0026:Upgrade required but not allowed - Connector: generic-jdbc-connector
	at org.apache.sqoop.repository.JdbcRepository$3.doIt(JdbcRepository.java:176)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:90)
	at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61)
	at org.apache.sqoop.repository.JdbcRepository.registerConnector(JdbcRepository.java:156)
	at org.apache.sqoop.connector.ConnectorManager.registerConnectors(ConnectorManager.java:214)
	... 27 more

Shutting down Sqoop server

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Wed, 09 Apr 2014 23:33:25 GMT

When you get that message, it generally means you should run the upgrade command. CM has made this one easy for you, so no need to figure out how to do the upgrade manually.

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Wed, 09 Apr 2014 23:45:01 GMT

Thanks again,

but how do I run this upgrade command in CM.

Any tips would be helpful.

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Wed, 09 Apr 2014 23:46:51 GMT

Click on Sqoop, then in the Actions dropdown menu on the right, you'll see an option to Upgrade Sqoop. If it's greyed out, that's because you need to stop Sqoop first.

Thanks,

DArren

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Thu, 10 Apr 2014 18:55:25 GMT

I am using CDH 4.6,

I don't see the upgrade in action

these all the choices I have start, stop,restart,rename,enter maintenance mode, create group.

Let me know If I am missing something.

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Thu, 10 Apr 2014 19:08:18 GMT

It looks like the sqoop upgrade action is only available in CM for CDH 5 clusters. Sorry for the confusion.

I'm not sure what triggered the need for a sqoop upgrade, but it seems like you'll need to use safety valves to configure sqoop.properties to perform this upgrade manually.

I'd be curious to know what you might have done to trigger a sqoop upgrade in CDH 4. CM currently assumes that's impossible.

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Thu, 10 Apr 2014 22:21:35 GMT

After the CM upgrade from 4.5 to 4.6, I get these errors, I am not able to start the sqoop2 service. So on the forum it is said to manually edit the sqoop.properties files.

I did the following

/etc/sqoop2
conf -> /etc/alternatives/sqoop2-conf
/etc/alternatives/sqoop2-conf -> /hdp2014poc/cloudera/parcel-repo/CDH-4.5.0-1.cdh4.5.0.p0.30/etc/sqoop2/conf.dist
sqoop.properties

org.apache.sqoop.connector.autoupgrade=true
org.apache.sqoop.framework.autoupgrade=true

even after updating the sqoop.properties, I still get the above error.

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Thu, 10 Apr 2014 22:55:10 GMT

CM-managed sqoop2 doesn't read from that file. Edit the safety valve for sqoop.properties instead.

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Thu, 10 Apr 2014 23:14:59 GMT

Thanks again,

but we are back to sqare one, I will repost the question again,

Problem desc:

I need to update the 2 properties in sqoop, but I am not able do it through sqoop.propeties files as CM will overide it, Its need to be done in the Sqoop Service Environment Safety Valve in CM,

But when I did it in the sqoop2 service, in the advanced configuration,to setup key/value pair for "Sqoop Service Environment Safety Valve", below is the format.

org.apache.sqoop.connector.autoupgrade=true

org.apache.sqoop.framework.autoupgrade=true

I get the below error,

Could not parse: Sqoop Service Environment Safety Valve : Could not parse parameter 'sqoop_env_safety_valve'. Was expecting: valid variable name. Input: connector.autoupgrade=true

If i just give it like this it works

autoupgrade=true

but I need the whole string. I need help, what is format?

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Fri, 11 Apr 2014 00:13:01 GMT

There are multiple safety valves for sqoop, and you are editing the wrong one. See my first reply.

You need to edit the one that will alter sqoop.properties, NOT the one that will alter the environment. Read the name and description carefully.

Re: CM update Sqoop Service Environment Safety Valve i

praveenhm — Fri, 11 Apr 2014 17:12:39 GMT

Thanks Darren,

Your are right there are couple of saftey valve, I was modifiying the wrong one.

I got it working.

Thanks again.

Praveen

Re: CM update Sqoop Service Environment Safety Valve i

Darren — Fri, 11 Apr 2014 17:46:28 GMT

Glad you got it working!