Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hue+Sqoop+custom query bug


Hue+Sqoop+custom query bug

New Contributor


I need to shedule sqoop import task using hue.

Original command looks like this:



sqoop-import     --connect jdbc:mysql://   --username username --password  password   --query 'SELECT *, CONCAT(col1, "-", col2, "-", UNIX_TIMESTAMP(col3)) AS rowkey FROM table WHERE $CONDITIONS'     --hbase-table table     --column-family cf     --hbase-row-key rowkey  --split-by id --check-column col3 --incremental append



The key problem, ofc, is custom query. At first I tried to use full command and get an error:


ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: 

I've found solution here, to put all arguments in special fields.



I did it, but when I tried to submit a workflow, I got this error:


Error submitting workflow sqoop-import

E0701: XML schema error, cvc-complex-type.2.4.a: Invalid content was found starting with element 'arg'. One of '{"uri:oozie:sqoop-action:0.2":file, "uri:oozie:sqoop-action:0.2":archive}' is expected.

Can you tell me how I can fix this, or advise some trick or another way to make this workflow work?

I use Hue 3.7.0 and CDH-5.4.2-1.



Re: Hue Sqoop custom query bug

Did you try to just put the full command like:

import --connect jdbc:mysql:// ....

Could you make sure you are not hitting and having an empty 'arg'?


Re: Hue Sqoop custom query bug

Expert Contributor

Using the command soley or args soley should work. The DTD for sqoop looks like this:

<xs:complexType name="ACTION">
            <xs:element name="job-tracker" type="xs:string" minOccurs="1" maxOccurs="1"/>
            <xs:element name="name-node" type="xs:string" minOccurs="1" maxOccurs="1"/>
            <xs:element name="prepare" type="sqoop:PREPARE" minOccurs="0" maxOccurs="1"/>
            <xs:element name="job-xml" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element name="configuration" type="sqoop:CONFIGURATION" minOccurs="0" maxOccurs="1"/>
                <xs:element name="command" type="xs:string" minOccurs="1" maxOccurs="1"/>
                <xs:element name="arg" type="xs:string" minOccurs="1" maxOccurs="unbounded"/>
            <xs:element name="file" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>
            <xs:element name="archive" type="xs:string" minOccurs="0" maxOccurs="unbounded"/>

Choice means it's either one or the other.

Don't have an account?
Coming from Hortonworks? Activate your account here