Invalid/conflicting GC options found error in Hive

a_gulshani — Wed, 16 Aug 2017 12:05:32 GMT

Hi All!

I'm running query at hive and getting the following error. Please help!

hive> select serialno, count(*) from cdr_data where hday='2017-08-01' group by serialno having count(*) >1;

Query ID = hdfs_20170816104340_764cc276-8015-4655-b153-a6bcf445a603 Total jobs = 1 Launching Job 1 out of 1 Dag submit failed due to Invalid TaskLaunchCmdOpts defined for Vertex Map 1 : Invalid/conflicting GC options found, cmdOpts="-server -Djava.net.preferIPv4Stack=true -Dhdp.version=2.6.0.3-8 -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+ResizeTLAB -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseParallelGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/ -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=INFO,CLA " stack trace: [org.apache.tez.dag.api.DAG.createDag(DAG.java:879), org.apache.tez.client.TezClientUtils.prepareAndCreateDAGPlan(TezClientUtils.java:678), org.apache.tez.client.TezClient.submitDAGSession(TezClient.java:444), org.apache.tez.client.TezClient.submitDAG(TezClient.java:411), org.apache.hadoop.hive.ql.exec.tez.TezTask.submit(TezTask.java:470), org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:188), org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160), org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89), org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1748), org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1494), org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1291), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1158), org.apache.hadoop.hive.ql.Driver.run(Driver.java:1148), org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:217), org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:169), org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:380), org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:740), org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:685), org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625), sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method), sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62), sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43), java.lang.reflect.Method.invoke(Method.java:498), org.apache.hadoop.util.RunJar.run(RunJar.java:233), org.apache.hadoop.util.RunJar.main(RunJar.java:148)] retrying... FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

Re: Invalid/conflicting GC options found error in Hive

jsensharma — Wed, 16 Aug 2017 12:16:28 GMT

@Gulshan Agivetova

Please check and share the values of the following parameters "tez.am.launch.cmd-opts" and "hive.tez.java.opts" they should not be conflicting. Specially the GC options.

Try this: You should get rid of the "-XX:+UseParallelGC" option (remove this option) from the "tez.am.launch.cmd-opts" and "hive.tez.java.opts" properties at the very first place and then restart it.

This is because "-XX:+UseG1GC and -XX:+UseParallelGC" Should never be used together."

Re: Invalid/conflicting GC options found error in Hive

a_gulshani — Wed, 16 Aug 2017 15:12:20 GMT

@Jay SenSharma Thank you very much! 🙂

It really helped! I've set -XX:+UseG1GC for hive.tez.java.opts and it works!

Thank you!

Re: Invalid/conflicting GC options found error in Hive

jsensharma — Wed, 16 Aug 2017 15:23:34 GMT

@Gulshan Agivetova

Good to know that setting the -XX:+UseG1GC for hive.tez.java.opts worked.

- It will be really great if you can mark this thread as "Accepted" (Answered) so that it will be very useful for HCC users to quickly browse for the solutions.

question Invalid/conflicting GC options found error in Hive in Archives of Support Questions (Read Only)

Invalid/conflicting GC options found error in Hive

Re: Invalid/conflicting GC options found error in Hive

Re: Invalid/conflicting GC options found error in Hive

Re: Invalid/conflicting GC options found error in Hive