Support Questions

Find answers, ask questions, and share your expertise

ERROR [main] acid.PreUpgradeTool (PreUpgradeTool.java:main(152)) - PreUpgradeTool failed java.lang.IllegalArgumentException: java.net.UnknownHostException

Explorer

I am trying to run the hive preupgrade tool by following the instructions below..
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-upgrade-major/content/prepare_hiv...

And I am seeing this exception.. Looking at the exception it looks like it's not able to resolve the logical name node.


	java.lang.IllegalArgumentException: java.net.UnknownHostException: mytestcluster

	at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:439)

	at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:321)

	at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176)

	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:696)

	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:636)

	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:160)

	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2796)

	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)

	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2830)

	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2812)

	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)

	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)

	at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.needsCompaction(PreUpgradeTool.java:417)

	at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getCompactionCommands(PreUpgradeTool.java:384)

	at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.getCompactionCommands(PreUpgradeTool.java:374)

	at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.prepareAcidUpgradeInternal(PreUpgradeTool.java:235)

	at org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool.main(PreUpgradeTool.java:149)
1 REPLY 1

Explorer

I was able to resolve this error by passing in the following in the classpath (in addition to the other classpath entries). It's now able to generate the compaction sql commands..

etc/hadoop/conf:/etc/hive/conf

Here is the full command:

$JAVA_HOME/bin/java -Djavax.security.auth.useSubjectCredsOnly=false -cp /etc/hadoop/conf:/etc/hive/conf:/usr/hdp/$STACK_VERSION/hive/lib/derby-10.10.2.0.jar:/usr/hdp/$STACK_VERSION/hive/lib/*:/usr/hdp/$STACK_VERSION/hadoop/*:/usr/hdp/$STACK_VERSION/hadoop/lib/*:/usr/hdp/$STACK_VERSION/hadoop-mapreduce/*:/usr/hdp/$STACK_VERSION/hadoop-mapreduce/lib/*:/usr/hdp/$STACK_VERSION/hadoop-hdfs/*:/usr/hdp/$STACK_VERSION/hadoop-hdfs/lib/*:/usr/hdp/$STACK_VERSION/hadoop/etc/hadoop/*:/tmp/hive-pre-upgrade-3.1.0.3.1.0.0-78.jar:/usr/hdp/$STACK_VERSION/hive/conf/conf.server org.apache.hadoop.hive.upgrade.acid.PreUpgradeTool