Member since
08-08-2013
109
Posts
19
Kudos Received
10
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
6570 | 07-28-2015 02:13 PM | |
79355 | 12-12-2014 07:28 AM | |
6167 | 11-19-2014 04:14 PM | |
10748 | 11-17-2014 10:35 AM | |
9426 | 08-05-2014 10:56 AM |
07-09-2014
11:37 AM
Warren, Glad that worked. I'm not really sure if Sqoop2 has a feature to do that. You might be able to reduce the number of mappers down to 1 using throttlers. When using Hue or the Sqoop2 client, the number of throttlers during the extraction phase and load phase can be numerically defined. If left blank or 0 is provided, then the execution framework may ignore it. If you throttle the number of extractors to 1, then you'll have 1 mapper perform the extraction. If you throttle the number of loaders to 1, you'll have at most 1 loader write to HDFS. If that doesn't work, your best bet is performing the concatenation yourself as you've described.
... View more
07-09-2014
10:21 AM
Could you try adding a column to partition by (that isn't of type uniqueidentifier)? If no partition column is provided, Sqoop will automatically split by the primary key, which is AssetId in this case. You should be able to split by text and date types worst case.
... View more
07-08-2014
12:46 PM
Hey there, I don't think there's a good way of doing this in Sqoop2 yet (https://issues.apache.org/jira/browse/SQOOP-1074). However, it seems the conversion is failing in the driver and not the connector. Could you provide the schema of your table?
... View more
07-08-2014
08:42 AM
1 Kudo
Hey there, Would you mind setting the verbosity option and running your job? set option --name verbose --value true Also, /var/log/sqoop2/sqoop.log should have errors. Could you please dig for any errors there and post them to this thread? -Abe
... View more
06-28-2014
02:16 PM
Unfortunately, that would require a custom output format, which is currently not supported in Sqoop 1. You might be able to get away with writing a custom storage format for HCatalog. This seems like an interesting feature to add in Sqoop2 though: https://issues.apache.org/jira/browse/SQOOP-1346.
... View more
06-24-2014
01:36 PM
1 Kudo
Also, it looks like https://issues.apache.org/jira/browse/SQOOP-1138 exists to address this concern.
... View more
06-24-2014
01:35 PM
You can set permissions through Hue via the file browser. There should be a Chmod/Chown button above the file listing. Are you logged in as warren.lastname user? Would you be able to run the job with the "verbose" option from the command line? Or, possibly, provide the Sqoop logs and task logs to see what happened?
... View more
06-24-2014
10:46 AM
By the way, what version of CDH are you using?
... View more
06-24-2014
10:40 AM
Hey there, It seems you shouldn't have to do this, but adding "--append" to your command might help. -Abe
... View more
06-24-2014
07:26 AM
This is a problem with Hue and SQLite in a multithreaded environment. A workaround to have this error be intermittent is to provide options='{"timeout": 30}' in the database section of your hue.ini. This should restrict locks to 30 seconds.
... View more
- « Previous
- Next »