Member since
10-05-2016
4
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
21556 | 01-23-2018 09:19 AM |
05-18-2018
08:58 AM
I am trying to export a directory using sqoop into SQL Server. I am having trouble with this one particular directory and getting the error found below. I have searched all over and tried many different iterations of my sqoop parameters to try to get it to work, but still cannot get it to work. If anyone has any suggestions they would be greatly appreaciated. ERROR from logs: 2018-05-18 08:41:20,378 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:rjkeen (auth:SIMPLE) cause:java.io.IOException: Can't export data, please check failed map task logs 2018-05-18 08:41:20,378 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Can't export data, please check failed map task logs at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:122) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.RuntimeException: Can't parse input data: 'http://tableau.abc.edu/views/SmartAlerts-Commercial/SmartAlerts?iframeSizedToWindow=true&:embed=y&:showAppBanner=false&:display_count=no&:showVizHome=no&Manager=JULY%20LIVINGSTON%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20&Customer=IN%20GRACE%20INITIATIVES%20LLC%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20&bilacct_k=1515009' at smart_alert.__loadFromFields(smart_alert.java:708) at smart_alert.parse(smart_alert.java:606) at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89) ... 10 more Caused by: java.util.NoSuchElementException at java.util.ArrayList$Itr.next(ArrayList.java:834) at smart_alert.__loadFromFields(smart_alert.java:703) ... 12 more Sqoop Export looks like: export --connect jdbc:sqlserver://ABC123;databaseName=test;integratedSecurity=false;applicationName=SQOOP; --username ********* --password ************** -m 1 --table smart_alert --columns "customer_account_id,alert_date,alert_level,fromdate,todate,expected_usage,actual_usage,percentdifference,alert_type,dashboard,dt" --input-fields-terminated-by ',' --input-lines-terminated-by '\n' --input-null-string "\\N" --input-null-non-string "\\N" -- --schema dbo Hive Table structure: CREATE EXTERNAL TABLE dev_smart_alert.commercial_usage_alert_test ( customer_account_id INT, alert_date STRING, alert_level INT, fromdate STRING, todate STRING, expected_usage DOUBLE, actual_usage DOUBLE, percentdifference DOUBLE, alert_type STRING, dashboard STRING ) PARTITIONED BY ( dt STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' WITH SERDEPROPERTIES ('field.delim'=',', 'line.delim'='\n', 'serialization.format'=',') STORED AS TEXTFILE LOCATION 'hdfs://nameservice1/development/warehouse/smart_alert/commercial_usage_alert_test' SQL Table Structure: [dbo].[smart_alert]( [customer_account_id] [int] NOT NULL, [alert_date] [date] NOT NULL, [alert_level] [smallint] NOT NULL, [fromdate] [date] NOT NULL, [todate] [date] NOT NULL, [expected_usage] [float] NOT NULL, [actual_usage] [float] NOT NULL, [percentdifference] [float] NOT NULL, [alert_type] [varchar](100) NOT NULL, [dashboard] [varchar](8000) NOT NULL, [dt] [date] NOT NULL )
... View more
Labels:
- Labels:
-
Apache Sqoop
05-02-2018
01:26 PM
Thanks Romain! Do you know what version of CDH will incorporate HUE 4.2 as part of the parcel? I see that Hue 4.2 is pretty recent. Is this something that will be incorporated into the CDH 6 release? We are currently running CDH 5.13. Thanks again! P.S. I always find your answers and videos super helpful.
... View more
04-26-2018
09:32 AM
We recently upgraded to CDH5.13 and started using HUE 4. I am still getting used to the navigation, but I cannot seem to find two options that were in HUE 3. I am looking for the Oozie Workflow editor and looking for the ability to copy a Oozie Workflow. I have found the Workflows in HUE 4 in the Documents section, but there does not seem to be a way to copy the workflows. In Hue 3 we were ablet o copy a workflow and in doing so it would create a new workspace, but keep the same structure which is ideal for us. I am at least looking for the ability to copy, but if there is a way to navigate to something like the picture below in Hue 4, that would be great.
... View more
Labels:
- Labels:
-
Cloudera Hue
01-23-2018
09:19 AM
2 Kudos
You need to put single quotes around your timestamps. Example) select * from dev_synchrophasor.si_co where `timestamp` between '2017-03-01 00:00:03.00' and '2017-03-01 00:00:10.0' Hope that helps!
... View more