Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sqoop Export - Can't parse input data

avatar
Explorer

I am trying to export a directory using sqoop into SQL Server. I am having trouble with this one particular directory and getting the error found below. I have searched all over and tried many different iterations of my sqoop parameters to try to get it to work, but still cannot get it to work. If anyone has any suggestions they would be greatly appreaciated.

 

ERROR from logs:

 

2018-05-18 08:41:20,378 WARN [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:rjkeen (auth:SIMPLE) cause:java.io.IOException: Can't export data, please check failed map task logs
2018-05-18 08:41:20,378 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Can't export data, please check failed map task logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:122)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.RuntimeException: Can't parse input data: 'http://tableau.abc.edu/views/SmartAlerts-Commercial/SmartAlerts?iframeSizedToWindow=true&:embed=y&:s...'
at smart_alert.__loadFromFields(smart_alert.java:708)
at smart_alert.parse(smart_alert.java:606)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:89)
... 10 more
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:834)
at smart_alert.__loadFromFields(smart_alert.java:703)
... 12 more

 

Sqoop Export looks like:

 

export

--connect jdbc:sqlserver://ABC123;databaseName=test;integratedSecurity=false;applicationName=SQOOP; --username ********* --password **************

-m
1
--table
smart_alert
--columns
"customer_account_id,alert_date,alert_level,fromdate,todate,expected_usage,actual_usage,percentdifference,alert_type,dashboard,dt"
--input-fields-terminated-by
','
--input-lines-terminated-by
'\n'
--input-null-string
"\\N"
--input-null-non-string
"\\N"
--
--schema
dbo

 

Hive Table structure:

CREATE EXTERNAL TABLE dev_smart_alert.commercial_usage_alert_test (   customer_account_id INT,   alert_date STRING,   alert_level INT,   fromdate STRING,   todate STRING,   expected_usage DOUBLE,   actual_usage DOUBLE,   percentdifference DOUBLE,   alert_type STRING,   dashboard STRING ) PARTITIONED BY (   dt STRING ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' WITH SERDEPROPERTIES ('field.delim'=',', 'line.delim'='\n', 'serialization.format'=',') STORED AS TEXTFILE LOCATION 'hdfs://nameservice1/development/warehouse/smart_alert/commercial_usage_alert_test'
 
SQL Table Structure:
[dbo].[smart_alert](
 [customer_account_id] [int] NOT NULL,
 [alert_date] [date] NOT NULL,
 [alert_level] [smallint] NOT NULL,
 [fromdate] [date] NOT NULL,
 [todate] [date] NOT NULL,
 [expected_usage] [float] NOT NULL,
 [actual_usage] [float] NOT NULL,
 [percentdifference] [float] NOT NULL,
 [alert_type] [varchar](100) NOT NULL,
 [dashboard] [varchar](8000) NOT NULL,
 [dt] [date] NOT NULL
)
1 REPLY 1

avatar
New Contributor

Hi, I am also facing the same issue, could you please help if this resolved for you.

 

Thanks,

Aravind