Member since
03-23-2015
1288
Posts
114
Kudos Received
98
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2521 | 06-11-2020 02:45 PM | |
4049 | 05-01-2020 12:23 AM | |
2129 | 04-21-2020 03:38 PM | |
2095 | 04-14-2020 12:26 AM | |
1718 | 02-27-2020 05:51 PM |
06-25-2019
06:00 PM
@punshi, Looks like you want to load data with dynamic partitions. My suggestion is to load the data into a temp table without partitions and then load them into another one with partitions: 1. create table tbl_id_part_temp (id int, year int) row format delimited fields terminated by ','; 2. load data inpath '/user/impala/data/id.txt' into table tbl_id_part_temp; 3. create table tbl_id_part (id int) partitioned by (year int) row format delimited fields terminated by ','; 4. insert into tbl_id_part partition (year) select id, year from tbl_id_part_temp; This is because LOAD DATA does not support dynamic partition loading at the moment: https://www.cloudera.com/documentation/enterprise/latest/topics/impala_load_data.html#load_data >>> When the LOAD DATA statement operates on a partitioned table, it always operates on one partition at a time. Specify the PARTITION clauses and list all the partition key columns, with a constant value specified for each. Cheers Eric
... View more
06-20-2019
10:53 PM
i am also getting the same error after fixing the JDBC driver issue, in Oozie server log, I noticed below: 2019-06-20 22:51:19,640 WARN org.apache.oozie.action.hadoop.HadoopTokenHelper: SERVER[xxxx-xxx.cloudera.com] USER[admin] GROUP[-] TOKEN[] APP[My Workflow] JOB[0000009-190620171825845-oozie-oozi-W] ACTION[0000009-190620171825845-oozie-oozi-W@sqoop-6891] An error happened while trying to get server principal. Getting it from service principal anyway. java.lang.IllegalArgumentException: Does not contain a valid host:port authority: yarnRM Can you check if you get the same error? I am still researching... Cheers Eric
... View more
06-20-2019
10:21 PM
@anis447 Can you please share the content of stderr.log for the launcher? I have tested in my 6.2 cluster, my job failed with below error in stderr.log: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver And I see the same message as yours in stdout.log: Fetching child yarn jobs tag id : oozie-b1d7e6d15cf45e6c89a06ad6c89e7109 No child applications found No child hadoop job is executed. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410) at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55) at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217) at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875) at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141) Caused by: java.lang.SecurityException: Intercepted System.exit(1) at org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57) at java.lang.Runtime.exit(Runtime.java:107) at java.lang.System.exit(System.java:971) at org.apache.sqoop.Sqoop.main(Sqoop.java:252) at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:214) at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:199) at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104) at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:51) ... 16 more Looks like your Sqoop job failed and not even reach the step to create Sqoop YARN jobs. Cheers Eric
... View more
06-19-2019
11:52 PM
hmmm, I suspect it is the space between Reference and ID, Oozie breaks it up into different parameters and ignoring the quotes you have. Can you try to use another field that does not have space in it? Cheers Eric
... View more
06-19-2019
11:00 PM
@Shashank_219, For jobs submitted in the same Hue session, they will have different application IDs. Can you please share what you observed, via screenshot? This might help to clarify your question. Cheers Eric
... View more
06-19-2019
04:13 PM
1 Kudo
Hi @Mogiking Hue keeps the query handler open so that it can get back to its result during pagination. This is a feature in Hue, not a bug. You can find that info from impala PROFILE under Query Timeline section, like below: Query Timeline Query submitted: 0ns (0) Planning finished: 170ms (170000854) Submit for admission: 172ms (172000864) Completed admission: 172ms (172000864) Ready to start on 2 backends: 172ms (172000864) All 2 execution backends (3 fragment instances) started: 176ms (176000884) Rows available: 422ms (422002119) First row fetched: 501ms (501002515) Unregister query: 509ms (509002555) You can see detailed information on when data is available. Cheers Eric
... View more
06-19-2019
04:05 PM
Where do you put the jar file? Is it under /usr/share/java?
... View more
06-19-2019
04:04 PM
Can you please share the workflow.xml contents for review? The new line "Reference" does not seem correct, so I want to see how it is setup in the XML file . Please also share the screenshot of the Sqoop job configuration in Hue for review. Cheers Eric
... View more
06-16-2019
05:35 PM
Hi, Thanks for sharing the output. From the error, looks like that Sqoop did not parse the command correctly. And I noticed below parameter: --merge-key 'Reference ID' Noticed the line break between Reference and ID? I am not sure if it is formatting issue in the forum or it was like this in your Sqoop job setup. Can you please double check on this? And make sure there is no new line between the two words? Cheers Eric
... View more
06-12-2019
11:40 PM
Hi @EricC Are you able to share some of the failed impala query PROFILE for review? It might tell some story from there. Cheers EricL
... View more