The query executes with map reduce engine and I get the desired result.  The error happens when I switch to spark engine. ... View more

The SQL is very simple.  Select col1, col2,col3,.............,coln from Table1 UNION ALL Select col1, col2,col3.............,coln from Table2 UNION ALL Select col1, col2,col3.............,coln from Table3 UNION ALL .......... UNION ALL Select col1, col2,col3.............,coln from Table13   When I break up the view into smaller chunks, they work.  Here is the sample of combinations that work and do not work.   Tables   1,2,3,4 Working                                 1,2,3,4,5 Not Working                                 1,2,3,5 Not Working                                 1,2,4,5 Not Working                                 1,3,4,5 Not Working                                 1,2,3,4,6 Not Working                                 1,2,3,4,7 Not Working                                 1,2,3,4,8 Not Working                                 1,5 Working                                 5,6 Working                                 5,6,7 Not Working                                 6,7 Working                                 6,7,8 Working                                 6,7,8,9 Working                                 6,7,8,9,10 Working                                 6,7,8,9,10,11 Working                                 6,7,8,9,10,11,12 Not Working                                 12,13 Working   These is a partial list of combinations, I have tried.   When it is not working, I get this stack trace.   Getting log thread is interrupted, since query is done! Error: Error while compiling statement: FAILED: SemanticException Exception when trying to remove partition predicates: fail to find child from parent (state=42000,code=40000) org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Exception when trying to remove partition predicates: fail to find child from parent         at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:239)         at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:225)         at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244)         at org.apache.hive.beeline.Commands.executeInternal(Commands.java:904)         at org.apache.hive.beeline.Commands.execute(Commands.java:1091)         at org.apache.hive.beeline.Commands.sql(Commands.java:987)         at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1082)         at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917)         at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:845)         at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482)         at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465)         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:497)         at org.apache.hadoop.util.RunJar.run(RunJar.java:221)         at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Exception when trying to remove partition predicates: fail to find child from parent         at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:385)         at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:136)         at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:206)        at org.apache.hive.service.cli.operation.Operation.run(Operation.java:327)         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:424)         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:401)         at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:260)         at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:500)         at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)         at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)         at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)         at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)         at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:746)         at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to remove partition predicates: fail to find child from parent         at org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:711)         at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.removeUnionOperators(GenSparkUtils.java:284)         at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:229)         at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:204)         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10109)         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9902)         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:223)         at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:223)         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:488)         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1274)         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1261)         at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:134)         ... 15 more   ... View more