I am trying to import an Oracle table using below sqoop command. The column REPORT_XML_DATA is of the form BLOB. When trying to export the below query, I am getting Error: java.io.IOException: SQLException in nextKeyValue Caused by: java.sql.SQLException: Invalid column type: getString not implemented for class oracle.jdbc.driver.T4CBlobAccessor sqoop import --connect jdbc:oracle:thin:@xxxx:3000:ABCD --username TEST --password-file /user/test/mypassword.pwd --query "select REPORT_XML_DATA from mydb.table_name WHERE \$CONDITIONS" --map-column-java REPORT_XML_DATA=String --target-dir /user/test/sqoop/table_name --m 1 --hive-import --hive-table table_name --fields-terminated-by '\0174' --hive-drop-import-delims --null-string '\\N' --null-non-string '\\N' Error: 17/07/30 11:17:07 INFO mapreduce.Job: Task Id : attempt_1501271060971_7271_m_000000_1, Status : FAILED Error: java.io.IOException: SQLException in nextKeyValue at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556) at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164) Caused by: java.sql.SQLException: Invalid column type: getString not implemented for class oracle.jdbc.driver.T4CBlobAccessor at oracle.jdbc.driver.Accessor.unimpl(Accessor.java:414) at oracle.jdbc.driver.BlobAccessor.getString(BlobAccessor.java:335) at oracle.jdbc.driver.OracleResultSetImpl.getString(OracleResultSetImpl.java:1297) at org.apache.sqoop.lib.JdbcWritableBridge.readString(JdbcWritableBridge.java:71) at com.cloudera.sqoop.lib.JdbcWritableBridge.readString(JdbcWritableBridge.java:61) at QueryResult.readFields(QueryResult.java:90) at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:244) ... View more

I have multiple queries in a hql file (say 10, every query ending with 😉 which I am running from a shell script. When a query in between fails (say query #5), the queries after 5 do not execute, and the hive job is completed. How can I do error handling to make sure that queries from 6 to 10 run even though query 5 fails? ... View more

Hi All, I am trying to create a table in Hive from a txt file using a shell script in this format. My t_cols.txt has data as below: id string, name string, city string, lpd timestamp I want to create hive table whose columns should be coming from this text file. This is how my hive query looks like: table_cols=`cat t_cols.txt` hive --hiveconf t_name=${table_cols} -e 'create table leap_frog_snapshot.LINKED_OBJ_TRACKING (\${hiveconf:t_name}) stored as orc tblproperties ("orc.compress"="SNAPPY") ; ' This is not working somehow. Can someone explain how to achieve this? ... View more

I am trying to run oozie workflow from Hue. I am trying to run a hive job in the workflow and that seem to be working but overall I am getting error in the oozie workflow. Error: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10096] I have a jdbc driver in hdfs and HDP 2.5.3 Hue version: 2.6.1-37 Log: 2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] Start action [0000005-170304185753616-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] [***0000005-170304185753616-oozie-oozi-W@:start:***]Action status=DONE 2017-03-06 15:47:46,557 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] [***0000005-170304185753616-oozie-oozi-W@:start:***]Action updated in DB! 2017-03-06 15:47:46,611 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@:start:] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@:start: 2017-03-06 15:47:46,611 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W 2017-03-06 15:47:46,627 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Start action [0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:47:48,278 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] checking action, hadoop job ID [job_1488578860595_0014] status [RUNNING] 2017-03-06 15:47:48,281 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] [***0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean***]Action status=RUNNING 2017-03-06 15:47:48,281 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] [***0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean***]Action updated in DB! 2017-03-06 15:47:48,287 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean 2017-03-06 15:48:51,019 INFO CallbackServlet:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] callback for action [0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] 2017-03-06 15:48:51,151 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Trying to get job [job_1488578860595_0014], attempt [1] 2017-03-06 15:48:51,239 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Hadoop Jobs launched : [job_1488578860595_0015] 2017-03-06 15:48:51,241 INFO HiveActionExecutor:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] action completed, external ID [job_1488578860595_0014] 2017-03-06 15:48:51,255 WARN HiveActionExecutor:523 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [10096] 2017-03-06 15:48:51,281 INFO ActionEndXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] ERROR is considered as FAILED for SLA 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] Start action [0000005-170304185753616-oozie-oozi-W@kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] [***0000005-170304185753616-oozie-oozi-W@kill***]Action status=DONE 2017-03-06 15:48:51,320 INFO ActionStartXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[eedc] GROUP[-] TOKEN[] APP[test] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] [***0000005-170304185753616-oozie-oozi-W@kill***]Action updated in DB! 2017-03-06 15:48:51,373 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@kill] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@kill 2017-03-06 15:48:51,374 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W 2017-03-06 15:48:51,374 INFO WorkflowNotificationXCommand:520 - SERVER[phcv-hddev01.corp.incresearch.com] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000005-170304185753616-oozie-oozi-W] ACTION[0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean] No Notification URL is defined. Therefore nothing to notify for job 0000005-170304185753616-oozie-oozi-W@daily_snapshot_clean workflow.xml <workflow-app name="test" xmlns="uri:oozie:workflow:0.4"> <start to="daily_snapshot_clean"/> <action name="daily_snapshot_clean"> <hive xmlns="uri:oozie:hive-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <script>/user/eedc/oozie/sbi/hive/daily/daily_snapshot_clean_sbi_master_start.hql</script> </hive> <ok to="end"/> <error to="kill"/> </action> <kill name="kill"> <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end"/> </workflow-app> ... View more