Reply
Explorer
Posts: 12
Registered: ‎06-15-2018

Sqoop Hcatlog ORC is successful in edge node but fails in Oozie

Below sqoop command works perfectly fine in edge node but it fails when I try to schedule it through Oozie. Is this a bug?

I'm using Oozie for parquet and spark schedules so I'm sure about workflow settings and other settings

 

sqoop import -D mapred.job.queue.name=xxxxxx  -D mapreduce.map.memory.mb=3072 -D mapreduce.map.java.opts=-Xmx3072m --connect "jdbc:oracle:thin:@abcd04-scan.sdi.company.com:xxxx/DB_SER01" --table TABLE --username USER --password-file path --split-by ID -m 1 --hcatalog-database hive_db --hcatalog-table TABLE_ORC --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")'

 

Error:

2018-10-17 14:20:15,658 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2018-10-17 14:20:15,723 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.6-cdh5.14.2
2018-10-17 14:20:15,733 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - mapred.job.queue.name is deprecated. Instead, use mapreduce.job.queuename
2018-10-17 14:20:15,808 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing arguments for import:
2018-10-17 14:20:15,808 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: as
2018-10-17 14:20:15,809 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: orc
2018-10-17 14:20:15,809 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: tblproperties
2018-10-17 14:20:15,809 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: ("orc.compress"="SNAPPY")'

<<< Invocation of Sqoop command completed <<<

No child hadoop job is executed.
Intercepting System.exit(1)

 

Announcements
New solutions