question How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run? in Archives of Support Questions (Read Only)

How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

mph — Thu, 11 Aug 2016 21:40:32 GMT

Hi,

When I run an insert into command through beeline Hive/Tez requests 2 containers. Once beeline reports that the row was successfully inserted in to the table I see that the job created (seen in the YARN Manager UI) is still running and holds on to one of the two containers, when I terminate beeline the job listed in the Manager UI then lists as completed.

Why is this happening and how can I change my hadoop configuration to stop this happening?

Thanks,

Mike

Re: How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

sunile_manjee — Thu, 11 Aug 2016 21:40:58 GMT

@mike harding check if you have tez reuse container turned on

tez.am.container.reuse.enabled=true

Configuration that specifies whether a container should be reused.

This allow other application to reuse tez containers to increase performance. turn it off if you are not interested in that functinoality.

Re: How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

sunile_manjee — Thu, 11 Aug 2016 23:57:14 GMT

what is

tez.am.session.min.held-containers

set to?

Re: How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

mph — Thu, 11 Aug 2016 23:58:20 GMT

@Sunile Manjee I checked this and it is false. The remaining container seems to be the application master. When I run the Hive jobs via MapReduce2 they complete fine, its just when they are run in Tez I see this behaviour.

Re: How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

mph — Fri, 12 Aug 2016 19:21:08 GMT

..I found that in my ambari settings this was not specified - on setting this to 0 and setting tez.session.am.dag.submit.timeout.secs to a smaller amount gave me the behaviour i was looking for.

Re: How do I stop beeline holding onto YARN containers after a job (performing a hive insert into) has successfully run?

iroberts — Mon, 12 Sep 2016 20:01:08 GMT

@mike harding to add to this, Tez by default first initializes an AM whereas MapReduce does so at submission only. This is the reason you see the behavior you describe. The tez container has a timeout setting as you stated and that will determine how long lived that initial AM is