Created 03-25-2016 04:58 PM
Hi, I am trying to run some simple enough queries on my files (250000 records) and I am receiving an out of memory message as follows:
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1458912652919_0002_1_00, diagnostics=[Task failed, taskId=task_1458912652919_0002_1_00_000006, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap spaceat org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:159)at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139).........and this continues for many lines.
I have tried researching this issue and I came across this post but I am lost as to how I can modify the settings discussed: https://azure.microsoft.com/en-us/blog/hive-memory-settings-resolve-out-of-memory-errors-using-azure...
Can anyone shed any light on this for me please? By the way, below is the query I was trying to run on my Hive table:
SELECT * FROM pigoutputhive SORT BY Score DESC LIMIT 10;
Many thanks in advance.
Created 03-26-2016 11:57 AM
So I finally figured out my problem, it had to do with my own impatience!
Although Hive said the query ran successfully and had not returned any results, it was still working away in the background and eventually spat back some results. Although the results were not correct that's for another day.
I changed the settings as per the linked article above but needed an Admin profile to do so and it eventually worked.
Created 03-25-2016 09:51 PM
I came across this article and changed the recommended settings but when I now run the above query I receive no errors or results even though the status is completed. Has anyone any other suggestions please?
Created 03-26-2016 11:57 AM
So I finally figured out my problem, it had to do with my own impatience!
Although Hive said the query ran successfully and had not returned any results, it was still working away in the background and eventually spat back some results. Although the results were not correct that's for another day.
I changed the settings as per the linked article above but needed an Admin profile to do so and it eventually worked.
Created 03-26-2016 12:08 PM
@John Garrigan thank you for posting your resolution. Much appreciated!
Created 04-16-2016 02:15 PM
how do you login as admin profile? I am using the Maria_dev, it is all grey out in config, I could not change anything