Created 07-20-2017 10:22 PM
I run a Tez query and got IllegalArgumentException. I notice this error only happens when I give date with empty data. When I change the date to a date with data, like '2017-03-01', the query works.
Am I missing any settings?
SQL:
select *
from t1
where date in ('2015-03-01', '2015-03-02', '2015-03-03')
group by date;
Stacktrace:
Map 1: -/- Map 2: -/- Map 3: -/- Map 5: -/- Reducer 4: 0/2
Status: Failed
Vertex failed, vertexName=Map 1, vertexId=vertex_1500433809286_8042_2_01, diagnostics=[Vertex vertex_1500433809286_8042_2_01 [Map 1] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: tcaef_aut
h_dtl_full initializer failed, vertex=vertex_1500433809286_8042_2_01 [Map 1], java.lang.IllegalArgumentException
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1307)
at java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1195)
at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:89)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:519)
at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:446)
at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:300)
at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:123)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
]
Vertex killed, vertexName=Reducer 4, vertexId=vertex_1500433809286_8042_2_04, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1500433809286_8042_2_04 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Map 3, vertexId=vertex_1500433809286_8042_2_03, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1500433809286_8042_2_03 [Map 3] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Map 2, vertexId=vertex_1500433809286_8042_2_02, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1500433809286_8042_2_02 [Map 2] killed/failed due to:OTHER_VERTEX_FAILURE]
Vertex killed, vertexName=Map 5, vertexId=vertex_1500433809286_8042_2_00, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1500433809286_8042_2_00 [Map 5] killed/failed due to:OTHER_VERTEX_FAILURE]
Created 07-21-2017 06:07 AM
could you share the DDL of the table t1.
Use this command to get the DDL.
show create table t1;
Created 07-21-2017 06:49 AM
The issue could be due to product limitations wrt Date and Tez vectorization. Try running the query with hive.vectorized.execution.enabled=false.