Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hue+Sqoop+custom query bug

Hue+Sqoop+custom query bug

New Contributor

Hello.

I need to shedule sqoop import task using hue.

Original command looks like this:

 

 

sqoop-import     --connect jdbc:mysql://10.0.0.0:3306/dbname   --username username --password  password   --query 'SELECT *, CONCAT(col1, "-", col2, "-", UNIX_TIMESTAMP(col3)) AS rowkey FROM table WHERE $CONDITIONS'     --hbase-table table     --column-family cf     --hbase-row-key rowkey  --split-by id --check-column col3 --incremental append

 

 

The key problem, ofc, is custom query. At first I tried to use full command and get an error:

 

ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: 

I've found solution here, to put all arguments in special fields.

sqoop_bug.jpg

 

I did it, but when I tried to submit a workflow, I got this error:

 

Error submitting workflow sqoop-import

E0701: XML schema error, cvc-complex-type.2.4.a: Invalid content was found starting with element 'arg'. One of '{"uri:oozie:sqoop-action:0.2":file, "uri:oozie:sqoop-action:0.2":archive}' is expected.

Can you tell me how I can fix this, or advise some trick or another way to make this workflow work?

I use Hue 3.7.0 and CDH-5.4.2-1.

Thanks!

2 REPLIES 2

Re: Hue Sqoop custom query bug

Did you try to just put the full command like:

import --connect jdbc:mysql://10.0.0.0:3306/dbname ....

Could you make sure you are not hitting
https://issues.cloudera.org/browse/HUE-2754 and having an empty 'arg'?

Romain

Highlighted

Re: Hue Sqoop custom query bug

Expert Contributor

Using the command soley or args soley should work. The DTD for sqoop looks like this:

<xs:complexType name="ACTION">
        <xs:sequence>
            <xs:element name="job-tracker" type="xs:string" minOccurs="1" maxOccurs="1"/>
            <xs:element name="name-node" type="xs:string" minOccurs="1" maxOccurs="1"/>
            <xs:element name="prepare" type="sqoop:PREPARE" minOccurs="0" maxOccurs="1"/>
            <xs:element name="job-xml" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element name="configuration" type="sqoop:CONFIGURATION" minOccurs="0" maxOccurs="1"/>
            <xs:choice>
                <xs:element name="command" type="xs:string" minOccurs="1" maxOccurs="1"/>
                <xs:element name="arg" type="xs:string" minOccurs="1" maxOccurs="unbounded"/>
            </xs:choice>
            <xs:element name="file" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element name="archive" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

Choice means it's either one or the other.