Member since
08-08-2013
339
Posts
132
Kudos Received
27
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
14830 | 01-18-2018 08:38 AM | |
1574 | 05-11-2017 06:50 PM | |
9195 | 04-28-2017 11:00 AM | |
3438 | 04-12-2017 01:36 AM | |
2841 | 02-14-2017 05:11 AM |
11-02-2013
12:22 PM
Hi Abe, thanks for your reply. I added the following properties to the "Sqoop Server Configuration Safety Valve for sqoop.properties" (after stopping service sqoop first): org.apache.sqoop.repository.schema.immutable=false org.apache.sqoop.connector.autoupgrade=true org.apache.sqoop.framework.autoupgrade=true The sqoop.properties file on the related server contains the entries, so they applied correctly. The error in the sqoop-log disappeared, but the main problem still exists => after clicking the Sqoop icon I receive a blank page (except heading "Sqoop Jobs", no wizard. The metadata upgrade worked fine, and I cannot detect any error message in one of the three log files sqoop-cmf-sqoop1-SQOOP_SERVER-hadoop-pg-2.cluster.log.out catalina.2013-11-02.log derbyrepo.log This is the part of the sqoop-cmf-... log file of upgrading the connectors => 2013-11-02 20:08:21,241 INFO org.apache.sqoop.repository.Repository: Upgrading metadata for connector: generic-jdbc-connector 2013-11-02 20:08:21,242 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-begin: 3, rollback: false 2013-11-02 20:08:21,393 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-commit: 3, rollback: false 2013-11-02 20:08:21,393 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-close: 2, rollback: false 2013-11-02 20:08:21,394 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-begin: 3, rollback: false 2013-11-02 20:08:21,494 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-commit: 3, rollback: false 2013-11-02 20:08:21,494 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-close: 2, rollback: false 2013-11-02 20:08:21,494 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-begin: 3, rollback: false 2013-11-02 20:08:21,858 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-commit: 3, rollback: false 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-close: 2, rollback: false 2013-11-02 20:08:21,859 INFO org.apache.sqoop.repository.Repository: Metadata upgrade finished for connector: generic-jdbc-connector 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-commit: 2, rollback: false 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-close: 1, rollback: false 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.connector.ConnectorManager: Registered connector: connector-generic-jdbc-connector:1:org.apache.sqoop.connector.jdbc.GenericJdbcConnector, Connection: Forms: form-connection:12:[input-connection.jdbcDriver:40:STRING:128, input-connection.connectionString:41:STRING:128, input-connection.username:42:STRING:40, input-connection.password:43:STRING:40, input-connection.jdbcProperties:44:MAP]Job type: IMPORTForms: form-table:13:[input-table.schemaName:45:STRING:50, input-table.tableName:46:STRING:50, input-table.sql:47:STRING:2000, input-table.columns:48:STRING:50, input-table.partitionColumn:49:STRING:50, input-table.partitionColumnNull:50:BOOLEAN, input-table.boundaryQuery:51:STRING:50]Job type: EXPORTForms: form-table:14:[input-table.schemaName:52:STRING:50, input-table.tableName:53:STRING:2000, input-table.sql:54:STRING:50, input-table.columns:55:STRING:50] 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-commit: 1, rollback: false 2013-11-02 20:08:21,859 DEBUG org.apache.sqoop.repository.JdbcRepositoryTransaction: Tx count-close: 0, rollback: false 2013-11-02 20:08:21,859 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2013-11-02 20:08:21,866 INFO org.apache.sqoop.connector.ConnectorManager: Connectors loaded: {generic-jdbc-connector={generic-jdbc-connector:org.apache.sqoop.connector.jdbc.GenericJdbcConnector:jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/sqoop2/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.2-cdh4.4.0.jar!/sqoopconnector.properties}} ... Some further ideas / suggestions ? Do you need some additional logs ? br...Gerd...
... View more
11-01-2013
02:46 AM
Does somebody have any hints for how to get to a working Sqoop wizard ?
... View more
10-31-2013
12:32 AM
Hi, issue is solved, I think I need to take some reading lessons 🙂 The entry for the CM was wrong, it has to be => deb [arch=amd64] http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
... View more
10-31-2013
12:29 AM
Hi, stupid me 😉 . Issue has been resolved...easily. I saw the error after evaluating the .list file again and again. The entry for the CM was wrong, it has to be: deb [arch=amd64] http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib br...Gerd...
... View more
10-31-2013
12:09 AM
Hi, I post this question here in addition to the CDH section, since I don't know if its CDH or CM related. I've installed a new Ubuntu12.04.03 64bit server vm (Virtualbox on Linux) and added the cloudera repos for CDH and CM in an own .list file, and added the repo key via "curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -" successfully. Just as described here: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Instal... Sadly the command "sudo apt-get update" produces errors => "" ... Hit http://security.ubuntu.com precise-security/main Translation-en Hit http://security.ubuntu.com precise-security/multiverse Translation-en Hit http://security.ubuntu.com precise-security/restricted Translation-en Hit http://security.ubuntu.com precise-security/universe Translation-en Err http://archive.cloudera.com precise-cdh4/contrib Sources 404 Not Found Err http://archive.cloudera.com precise-cdh4/contrib amd64 Packages 404 Not Found Err http://archive.cloudera.com precise-cdh4/contrib i386 Packages 404 Not Found Ign http://archive.cloudera.com precise-cdh4/contrib Translation-en Ign http://archive.cloudera.com precise-cdh4/contrib Translation-en W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/source/Sources 404 Not Found W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/binary-amd64/Pack... 404 Not Found W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/binary-i386/Packa... 404 Not Found E: Some index files failed to download. They have been ignored, or old ones used instead. "" me@ubuntu:~$ cat /etc/apt/sources.list.d/cloudera.list deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib What's going wrong here ?!?!
... View more
Labels:
10-29-2013
02:02 PM
Hi, I've installed a new Ubuntu12.04.03 64bit server vm and added the cloudera repos for CDH and CM in an own .list file => /etc/apt/sources.list.d/cloudera.list contains: deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib and added the repo key via "curl -s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add -" successfully. Just as described here: http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_install_path_B.html Sadly the command "sudo apt-get update" produces errors => "" ... Hit http://security.ubuntu.com precise-security/main Translation-en Hit http://security.ubuntu.com precise-security/multiverse Translation-en Hit http://security.ubuntu.com precise-security/restricted Translation-en Hit http://security.ubuntu.com precise-security/universe Translation-en Err http://archive.cloudera.com precise-cdh4/contrib Sources 404 Not Found Err http://archive.cloudera.com precise-cdh4/contrib amd64 Packages 404 Not Found Err http://archive.cloudera.com precise-cdh4/contrib i386 Packages 404 Not Found Ign http://archive.cloudera.com precise-cdh4/contrib Translation-en Ign http://archive.cloudera.com precise-cdh4/contrib Translation-en W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/source/Sources 404 Not Found W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/binary-amd64/Packages 404 Not Found W: Failed to fetch http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/dists/precise-cdh4/contrib/binary-i386/Packages 404 Not Found E: Some index files failed to download. They have been ignored, or old ones used instead. "" me@ubuntu:~$ cat /etc/apt/sources.list.d/cloudera.list deb [arch=amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib deb http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm/ precise-cdh4 contrib What's going wrong here ?!?!
... View more
Labels:
10-25-2013
05:21 AM
just out of curiosity I tried it in the Cloudera quickstart vm. Downloaded the latest 4.4 vmware image, logged into Hue as user "Cloudera" and clicked on the Sqoop icon. Same result => blank page, no wizard to create jobs etc...
... View more
10-25-2013
04:59 AM
I set the property accordingly in both files /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/etc/hue/hue.ini /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/share/hue/desktop/conf/hue.ini and restarted Hue as well as Sqoop, but the problem still exists. I assume the error (while restarting sqoop service) " Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0026:Upgrade required but not allowed - Connector: generic-jdbc-connector ... 2013-10-25 13:54:11,672 INFO org.apache.sqoop.core.SqoopServer: Shutting down Sqoop server 2013-10-25 13:54:11,675 TRACE org.apache.sqoop.framework.JobManager: Begin submission engine manager destroy " on the sqoop server is causing the troubles. Any hints regarding this ?!?!
... View more
10-24-2013
11:38 AM
Hi Romain, thanks for answering, the page you referred to is great. It's exactly the page I looked at before and thought the Hue version could cause the problem, since this page mentions Hue3. Nevertheless, I checked the logs, and......gotcha, here's the error => [24/Oct/2013 11:07:21 -0800] exception ERROR Could not connect to sqoop server: 111 (Connection refused) [24/Oct/2013 11:07:21 -0800] http_client DEBUG GET http://localhost:12000/sqoop/v1/job/all [24/Oct/2013 11:07:21 -0800] access INFO 10.147.x.y admin - "GET /sqoop/api/jobs/ HTTP/1.0" I'm wondering why Hue tries to connect to localhost ?!?! This will fail "by design" since sqoop is running on a different host. How can I modifiy that setting ? If I open "http://hadoop-pg-2:12000" I get the response, "Apache Sqoop ROOT", thereby I assume Sqoop is running correctly. UPDATE 1: but somehow wired, requesting "http://hadoop-pg-2:12000/sqoop/v1/job/all" returns a 404 "The requested resource () is not available." UPDATE 2: I'm somehow confused . The CM manager services overview tells me that service "sqoop" is up and running. I dived into /var/log/sqoop2 folder on the sqoop-server "hadoop-pg-2" and the file "sqoop-cmf-sqoop1-SQOOP_SERVER-hadoop-pg-2.cluster.log.out" shows => 2013-10-24 20:50:47,127 INFO org.apache.sqoop.core.SqoopServer: Booting up Sqoop server 2013-10-24 20:50:47,136 INFO org.apache.sqoop.core.PropertiesConfigurationProvider: Starting config file poller thread 2013-10-24 20:50:47,138 INFO org.apache.sqoop.audit.AuditLoggerManager: Begin audit logger manager initialization 2013-10-24 20:50:47,139 INFO org.apache.sqoop.audit.AuditLoggerManager: Audit logger manager initialized: OK 2013-10-24 20:50:47,141 INFO org.apache.sqoop.repository.RepositoryManager: Setting system properties: {derby.stream.error.file=/var/log/sqoop2/derbyrepo.log} 2013-10-24 20:50:47,153 INFO org.apache.sqoop.repository.JdbcRepositoryContext: [repo-ctx] handler=org.apache.sqoop.repository.derby.DerbyRepositoryHandler, conn-url=jdbc:derby:/var/lib/sqoop2/repository/db;create=true, driver=org.apache.derby.jdbc.EmbeddedDriver, user=null, password=*****, jdbc-props={}, tx-isolation=READ_COMMITTED, max-conn=10 2013-10-24 20:50:47,603 INFO org.apache.sqoop.repository.derby.DerbyRepositoryHandler: DerbyRepositoryHandler initialized. 2013-10-24 20:50:47,605 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepositoryProvider initialized 2013-10-24 20:50:47,605 INFO org.apache.sqoop.repository.JdbcRepositoryProvider: JdbcRepository initialized. 2013-10-24 20:50:47,605 INFO org.apache.sqoop.repository.RepositoryManager: Creating or upgrading on disk structures if necessary 2013-10-24 20:50:48,008 INFO org.apache.sqoop.repository.JdbcRepository: Creating repository schema objects 2013-10-24 20:50:48,722 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2013-10-24 20:50:48,750 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction commit 2013-10-24 20:50:48,750 INFO org.apache.sqoop.repository.RepositoryManager: Repository initialized: OK 2013-10-24 20:50:48,753 INFO org.apache.sqoop.connector.ConnectorManager: Connector config urls: [jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/sqoop2/webapps/sqoop/WEB-INF/lib/sqoop-connector-generic-jdbc-1.99.2-cdh4.4.0.jar!/sqoopconnector.properties] 2013-10-24 20:50:48,832 INFO org.apache.sqoop.connector.ConnectorHandler: Connector [org.apache.sqoop.connector.jdbc.GenericJdbcConnector] initialized. 2013-10-24 20:50:48,926 WARN org.apache.sqoop.repository.JdbcRepositoryTransaction: Marking transaction for rollback 2013-10-24 20:50:48,927 INFO org.apache.sqoop.repository.JdbcRepositoryTransaction: Attempting transaction roll-back 2013-10-24 20:50:48,928 ERROR org.apache.sqoop.core.SqoopServer: Server startup failure org.apache.sqoop.common.SqoopException: CONN_0007:Connector registration failed at org.apache.sqoop.connector.ConnectorManager.registerConnectors(ConnectorManager.java:234) at org.apache.sqoop.connector.ConnectorManager.initialize(ConnectorManager.java:195) at org.apache.sqoop.core.SqoopServer.initialize(SqoopServer.java:51) at org.apache.sqoop.server.ServerInitializer.contextInitialized(ServerInitializer.java:36) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4206) at org.apache.catalina.core.StandardContext.start(StandardContext.java:4705) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:799) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:943) at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:778) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:504) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324) at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065) at org.apache.catalina.core.StandardHost.start(StandardHost.java:840) at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057) at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463) at org.apache.catalina.core.StandardService.start(StandardService.java:525) at org.apache.catalina.core.StandardServer.start(StandardServer.java:754) at org.apache.catalina.startup.Catalina.start(Catalina.java:595) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414) Caused by: org.apache.sqoop.common.SqoopException: JDBCREPO_0026:Upgrade required but not allowed - Connector: generic-jdbc-connector at org.apache.sqoop.repository.JdbcRepository$3.doIt(JdbcRepository.java:176) at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:90) at org.apache.sqoop.repository.JdbcRepository.doWithConnection(JdbcRepository.java:61) at org.apache.sqoop.repository.JdbcRepository.registerConnector(JdbcRepository.java:156) at org.apache.sqoop.connector.ConnectorManager.registerConnectors(ConnectorManager.java:214) ... 27 more 2013-10-24 20:50:48,935 INFO org.apache.sqoop.core.SqoopServer: Shutting down Sqoop server Something happened to the "generic-jdbc-connector", where does it come from ? A apt-get update && apt-get upgrade didn't update/-grade anything.
... View more
10-24-2013
06:27 AM
Hi, I recently wanted to use the Sqoop UI in Hue to import data from mysql to HDFS, but after clicking the Sqoop2 icon I just got a blank page (except the heading "Sqoop Jobs"). Yes, of course, there aren't any jobs to display, but I expected the wizard to be displayed for creating jobs...?!?! Can I use Sqoop2 UI in Hue 2.5 at all, or do I need Hue 3 ? thanks in advance...Gerd... ============ env.: Hue 2.5.0 / sqoop2 / CDH4.4 / cluster managed by CM 4.6.2
... View more
Labels:
- Labels:
-
Apache Sqoop
-
Cloudera Hue
-
Cloudera Manager
-
HDFS
- « Previous
- Next »