Member since
06-26-2019
26
Posts
1
Kudos Received
0
Solutions
03-22-2020
07:26 AM
have you found a solution? I'm facing the same issue. I tried using /hive-standalone-metastore-3.1.2000.7.0.3.0-79.jar and hive-metastore-2.1.1-cdh6.2.0.jar, but all of them seemed not working.
... View more
12-26-2019
10:55 PM
Hi mike, did you try that? I'm also going to upgrade Hive with CDH 6.2
... View more
12-26-2019
10:39 PM
I'm using CDH 6.2.0. and I found the version of Hive is 2.1.1.
What I want is to upgrade Hive to 3.0 and keep all the Hive data (keep the metastore is enough?), but I can't find any document for upgrading hive in CDH?
Is there any document or tutorial for that? Thanks a lot.
Kind regards.
... View more
Labels:
- Labels:
-
Apache Atlas
-
Apache Hive
12-24-2019
12:35 AM
Thanks a lot, I take a try.
... View more
12-23-2019
06:10 PM
I simply set a new IP address to the server in my cluster by setting the DHCP in my router.
But, after restart the cluster, CDH is still connecting the old IP address, so that it can't get connected.
So, where can I update the IP address?
... View more
Labels:
- Labels:
-
Cloudera Essentials
08-29-2019
08:12 PM
Hi Michalis: I solved this by re-run the command: sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm mypassword This problem is due to my misunderstanding for scm_prepare_database.sh. Thanks a lot for your help. Kind regards
... View more
08-29-2019
07:04 PM
Hi Michalis: I just found that, I apologize for my late reply. That time I cleaned all metastore and recreated the database in MySQL and it worked. Recently however, this error happend again, as I restarted the cloudera-scm-server and cloudera-scm-agent By following your instruction, I found an important information: the CM_VERSION is in scm database!! mysql> SELECT table_name, table_schema AS dbname
-> FROM INFORMATION_SCHEMA.TABLES
-> WHERE table_name='CM_VERSION';
+------------+--------+
| table_name | dbname |
+------------+--------+
| CM_VERSION | scm |
+------------+--------+
1 row in set (2.11 sec) I don't know why it's gonna find in metastore database instead of scm. Could you give me some help, Thanks. Kind regards
... View more
07-23-2019
06:36 PM
clear explanation! Thanks bgooley.
... View more
07-22-2019
11:37 PM
@bgooley I have figured that out. Cause I changed the hive-site.xml in /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive/conf. Below is the wrong config. After I changed it back to the default config, it worked ! <?xml version="1.0" encoding="UTF-8"?>
<!--Autogenerated by Cloudera Manager-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://device1:3306/metastore?useUnicode=true&characterEncoding=UTF-8</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>mypassword</value>
</property>
<property>
<name>hive.metastore.try.direct.sql.ddl</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.try.direct.sql</name>
<value>true</value>
</property>
<property>
<name>datanucleus.schema.autoCreateAll</name>
<value>false</value>
</property>
<property>
<name>datanucleus.metadata.xml.validate</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>true</value>
</property>
<property>
<name>datanucleus.autoStartMechanism</name>
<value>SchemaTable</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
<property>
<name>hive.warehouse.subdir.inherit.perms</name>
<value>true</value>
</property>
<property>
<name>hive.server2.logging.operation.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>/var/log/hive/operation_logs</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>-1</value>
</property>
<property>
<name>hive.exec.reducers.bytes.per.reducer</name>
<value>67108864</value>
</property>
<property>
<name>hive.exec.copyfile.maxsize</name>
<value>33554432</value>
</property>
<property>
<name>hive.exec.reducers.max</name>
<value>1099</value>
</property>
<property>
<name>hive.metastore.execute.setugi</name>
<value>true</value>
</property>
<property>
<name>hive.support.concurrency</name>
<value>true</value>
</property>
<property>
<name>hive.zookeeper.quorum</name>
<value>device1</value>
</property>
<property>
<name>hive.zookeeper.client.port</name>
<value>2181</value>
</property>
<property>
<name>hive.zookeeper.namespace</name>
<value>hive_zookeeper_namespace_hive</value>
</property>
<property>
<name>hive.metastore.server.min.threads</name>
<value>200</value>
</property>
<property>
<name>hive.metastore.server.max.threads</name>
<value>100000</value>
</property>
<property>
<name>hive.cluster.delegation.token.store.class</name>
<value>org.apache.hadoop.hive.thrift.MemoryTokenStore</value>
</property>
<property>
<name>hive.metastore.fshandler.threads</name>
<value>15</value>
</property>
<property>
<name>hive.metastore.event.listeners</name>
<value></value>
</property>
<property>
<name>hive.metastore.server.max.message.size</name>
<value>196188569</value>
</property>
<property>
<name>hive.service.metrics.file.location</name>
<value>/var/log/hive/metrics-hivemetastore/metrics.log</value>
</property>
<property>
<name>hive.metastore.metrics.enabled</name>
<value>true</value>
</property>
<property>
<name>hive.service.metrics.file.frequency</name>
<value>30000</value>
</property>
<property>
<name>hadoop.security.credential.provider.path</name>
<value>localjceks://file//var/run/cloudera-scm-agent/process/47-hive-metastore-validate/creds.localjceks</value>
</property>
</configuration> I compared the default config to the wrong config, default config use this property to connect metastore, while I use "javax.jdo.option.ConnectionURL" to connect metastore. <property> <name>hive.metastore.uris</name> <value>thrift://device1:9083</value> </property> Thanks a lot for your help.
... View more
07-22-2019
10:32 PM
@bgooley I'm new to CM and I don't know what the machanism of CM behind, I just followed the document to install. In my cluster, there are two servers device1(master) and device2 . I can get the result properly on device2's hive command-line by: show databases; However, it can not run successfully on device1, and error log is metioned above. Then as regard to the gatway you mentioned, I found status of Gateway is None, is it normal? Thanks for your reply.
... View more
07-22-2019
08:35 PM
@bgooley as one of the log said: Caused by: java.lang.RuntimeException: Error getting metastore password: null should I write down my metastore password in hive-site.xml? I checked the config file in `/run/cloudera-scm-agent/process/81-hive-HIVEMETASTORE/hive-site.xml` the part of config is like this, I'm not sure if this is ok, as the password is '******', should I replace it with the plaintext password? <property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>********</value>
</property>
... View more
07-22-2019
08:20 PM
I can't even run show databases in hive command-line. I turned on debug mode can try again, it turned out as below: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1773) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:394) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:338) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:318) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:360) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:356) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:768) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:711) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1425) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1493) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1339) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1328) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:836) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:772) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:699) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.util.RunJar.run(RunJar.java:313) [hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:227) [hadoop-common-3.0.0-cdh6.2.0.jar:?]
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.lang.RuntimeException: Error getting metastore password: null
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:571) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.io.IOException
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:965) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.io.IOException: Configuration problem with provider path.
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2272) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:865) ~[sunjce_provider.jar:1.8.0_112]
at java.security.KeyStore.load(KeyStore.java:1445) ~[?:1.8.0_121]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:322) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:58) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:50) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider$Factory.createProvider(LocalJavaKeyStoreProvider.java:177) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2253) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:865) ~[sunjce_provider.jar:1.8.0_112]
at java.security.KeyStore.load(KeyStore.java:1445) ~[?:1.8.0_121]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:322) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:58) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:50) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider$Factory.createProvider(LocalJavaKeyStoreProvider.java:177) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2253) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 32 more
2019-07-23T11:06:10,489 WARN [5b430558-3bb5-4632-8b6a-649bbc40138b main] ql.Driver: Caught exception attempting to write metadata call information org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:242) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:394) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:338) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:318) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:360) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:356) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:768) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:711) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1425) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1493) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1339) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1328) [hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:239) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:187) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:409) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:836) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:772) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:699) [hive-cli-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.util.RunJar.run(RunJar.java:313) [hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:227) [hadoop-common-3.0.0-cdh6.2.0.jar:?]
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1773) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.lang.RuntimeException: Error getting metastore password: null
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:571) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.io.IOException
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:965) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.io.IOException: Configuration problem with provider path.
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2272) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_121]
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7034) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_121]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_121]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_121]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_121]
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1771) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3815) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3867) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3847) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:4101) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:254) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:865) ~[sunjce_provider.jar:1.8.0_112]
at java.security.KeyStore.load(KeyStore.java:1445) ~[?:1.8.0_121]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:322) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:58) ~[hadoop-common-3.0.0-cdh6.2.0.jar:?]
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:237) ~[hive-exec-2.1.1-cdh6.2.0.jar:2.1.1-cdh6.2.0]
... 23 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed I think there is some problem with the metastore (I'm using mysql), but don't know how to fix it.
... View more
07-22-2019
07:52 PM
Hi @bgooley I can use CM web UI to manipulate . But I also need to use hive command-line in bash environment.
... View more
07-22-2019
03:32 AM
The reason why I execute "hive --service metastore" is that there was an Exception when I run hive -e "LOAD ..": FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
... View more
07-22-2019
03:24 AM
When I run: hive --service metastore, it raised errors as below /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 24: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/hbase-hadoop2-compat.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 25: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/lib/htrace-core.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 26: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/hbase-common.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 27: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/hbase-server.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 28: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/hbase-hadoop-compat.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hive/conf/hive-env.sh: line 29: /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase/hbase-protocol.jar: Permission denied
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hadoop/libexec/hadoop-functions.sh: line 2331: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_USER: bad substitution
/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/bin/../lib/hadoop/libexec/hadoop-functions.sh: line 2426: HADOOP_ORG.APACHE.HADOOP.HBASE.UTIL.GETJAVAPROPERTY_OPTS: bad substitution
Starting Hive Metastore Server
WARNING: Use "yarn jar" to launch YARN applications.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
java.lang.RuntimeException: Error getting metastore password: null
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:571)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7028)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7023)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:7281)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7208)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
Caused by: java.io.IOException
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:965)
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566)
... 22 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959)
... 23 more
Caused by: java.io.IOException: Configuration problem with provider path.
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2272)
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191)
... 28 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:865)
at java.security.KeyStore.load(KeyStore.java:1445)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:322)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:58)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:50)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider$Factory.createProvider(LocalJavaKeyStoreProvider.java:177)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2253)
... 29 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
... 38 more
Exception in thread "main" java.lang.RuntimeException: Error getting metastore password: null
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:571)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:298)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:687)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:653)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:647)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:716)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:419)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7028)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:7023)
at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:7281)
at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7208)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:313)
at org.apache.hadoop.util.RunJar.main(RunJar.java:227)
Caused by: java.io.IOException
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:965)
at org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:566)
... 22 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.hive.shims.Hadoop23Shims.getPassword(Hadoop23Shims.java:959)
... 23 more
Caused by: java.io.IOException: Configuration problem with provider path.
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2272)
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2191)
... 28 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:865)
at java.security.KeyStore.load(KeyStore.java:1445)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:322)
at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:58)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider.<init>(LocalJavaKeyStoreProvider.java:50)
at org.apache.hadoop.security.alias.LocalJavaKeyStoreProvider$Factory.createProvider(LocalJavaKeyStoreProvider.java:177)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2253)
... 29 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
... 38 more Who can give me some hint, Thanks.
... View more
Labels:
- Labels:
-
Hive
07-18-2019
02:17 AM
1 Kudo
I solved it by cleaning the /dfs/nn and /dfs/dn because once I installed hdfs, and there were some data existed in the hdfs.
... View more
07-18-2019
01:23 AM
This is the stderr.log [18/Jul/2019 15:52:04 +0000] 2807 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/hdfs/hdfs.sh format-namenode cluster19
[18/Jul/2019 15:52:04 +0000] 2807 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 2811
[18/Jul/2019 15:52:04 +0000] 2812 MainThread redactor INFO Re-exec redactor: /opt/cloudera/cm-agent/bin/cm redactor --fds 3 5
[18/Jul/2019 15:52:04 +0000] 2812 MainThread redactor INFO Started redactor
2019年 07月 18日 星期四 15:52:04 CST
+ source_parcel_environment
+ '[' '!' -z /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/meta/cdh_env.sh ']'
+ OLD_IFS='
'
+ IFS=:
+ SCRIPT_ARRAY=($SCM_DEFINES_SCRIPTS)
+ DIRNAME_ARRAY=($PARCEL_DIRNAMES)
+ IFS='
'
+ COUNT=1
++ seq 1 1
+ for i in `seq 1 $COUNT`
+ SCRIPT=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/meta/cdh_env.sh
+ PARCEL_DIRNAME=CDH-6.2.0-1.cdh6.2.0.p0.967373
+ . /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/meta/cdh_env.sh
++ CDH_DIRNAME=CDH-6.2.0-1.cdh6.2.0.p0.967373
++ export CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ CDH_HADOOP_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ export CDH_MR1_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-0.20-mapreduce
++ CDH_MR1_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-0.20-mapreduce
++ export CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs
++ CDH_HDFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs
++ export CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-httpfs
++ CDH_HTTPFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-httpfs
++ export CDH_MR2_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce
++ CDH_MR2_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce
++ export CDH_YARN_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn
++ CDH_YARN_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn
++ export CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase
++ CDH_HBASE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase
++ export CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/zookeeper
++ CDH_ZOOKEEPER_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/zookeeper
++ export CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive
++ CDH_HIVE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive
++ export CDH_HUE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue
++ CDH_HUE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hue
++ export CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/oozie
++ CDH_OOZIE_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/oozie
++ export CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ CDH_HUE_PLUGINS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ export CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/flume-ng
++ CDH_FLUME_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/flume-ng
++ export CDH_PIG_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/pig
++ CDH_PIG_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/pig
++ export CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive-hcatalog
++ CDH_HCAT_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hive-hcatalog
++ export CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/sentry
++ CDH_SENTRY_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/sentry
++ export JSVC_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils
++ JSVC_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils
++ export CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/bin/hadoop
++ CDH_HADOOP_BIN=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/bin/hadoop
++ export CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/impala
++ CDH_IMPALA_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/impala
++ export CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/solr
++ CDH_SOLR_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/solr
++ export CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase-solr
++ CDH_HBASE_INDEXER_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hbase-solr
++ export SEARCH_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/search
++ SEARCH_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/search
++ export CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark
++ CDH_SPARK_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/spark
++ export WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ WEBHCAT_DEFAULT_XML=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/etc/hive-webhcat/conf.dist/webhcat-default.xml
++ export CDH_KMS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-kms
++ CDH_KMS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-kms
++ export CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/parquet
++ CDH_PARQUET_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/parquet
++ export CDH_AVRO_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/avro
++ CDH_AVRO_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/avro
++ export CDH_KAFKA_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kafka
++ CDH_KAFKA_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kafka
++ export CDH_KUDU_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu
++ CDH_KUDU_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/kudu
+ locate_cdh_java_home
+ '[' -z '' ']'
+ '[' -z /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils ']'
+ local BIGTOP_DETECT_JAVAHOME=
+ for candidate in "${JSVC_HOME}" "${JSVC_HOME}/.." "/usr/lib/bigtop-utils" "/usr/libexec"
+ '[' -e /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils/bigtop-detect-javahome ']'
+ BIGTOP_DETECT_JAVAHOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils/bigtop-detect-javahome
+ break
+ '[' -z /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils/bigtop-detect-javahome ']'
+ . /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/bigtop-utils/bigtop-detect-javahome
++ BIGTOP_DEFAULTS_DIR=/etc/default
++ '[' -n /etc/default -a -r /etc/default/bigtop-utils ']'
++ JAVA11_HOME_CANDIDATES=('/usr/java/jdk-11' '/usr/lib/jvm/jdk-11' '/usr/lib/jvm/java-11-oracle')
++ OPENJAVA11_HOME_CANDIDATES=('/usr/lib/jvm/jdk-11' '/usr/lib64/jvm/jdk-11')
++ JAVA8_HOME_CANDIDATES=('/usr/java/jdk1.8' '/usr/java/jre1.8' '/usr/lib/jvm/j2sdk1.8-oracle' '/usr/lib/jvm/j2sdk1.8-oracle/jre' '/usr/lib/jvm/java-8-oracle')
++ OPENJAVA8_HOME_CANDIDATES=('/usr/lib/jvm/java-1.8.0-openjdk' '/usr/lib/jvm/java-8-openjdk' '/usr/lib64/jvm/java-1.8.0-openjdk' '/usr/lib64/jvm/java-8-openjdk')
++ MISCJAVA_HOME_CANDIDATES=('/Library/Java/Home' '/usr/java/default' '/usr/lib/jvm/default-java' '/usr/lib/jvm/java-openjdk' '/usr/lib/jvm/jre-openjdk')
++ case ${BIGTOP_JAVA_MAJOR} in
++ JAVA_HOME_CANDIDATES=(${JAVA8_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]} ${JAVA11_HOME_CANDIDATES[@]} ${OPENJAVA11_HOME_CANDIDATES[@]})
++ '[' -z '' ']'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/java/jdk1.8*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/java/jre1.8*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/j2sdk1.8-oracle*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/j2sdk1.8-oracle/jre*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/java-8-oracle*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/Library/Java/Home*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/java/default*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/default-java*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/java-openjdk*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd '/usr/lib/jvm/jre-openjdk*'
++ for candidate_regex in ${JAVA_HOME_CANDIDATES[@]}
+++ ls -rvd /usr/lib/jvm/java-1.8.0-openjdk-amd64
++ for candidate in `ls -rvd ${candidate_regex}* 2>/dev/null`
++ '[' -e /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java ']'
++ export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
++ JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
++ break 2
+ get_java_major_version JAVA_MAJOR
+ '[' -z /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java ']'
++ /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -version
+ local 'VERSION_STRING=openjdk version "1.8.0_162"
OpenJDK Runtime Environment (build 1.8.0_162-8u162-b12-1-b12)
OpenJDK 64-Bit Server VM (build 25.162-b12, mixed mode)'
+ local 'RE_JAVA=[java|openjdk][[:space:]]version[[:space:]]\"1\.([0-9][0-9]*)\.?+'
+ [[ openjdk version "1.8.0_162"
OpenJDK Runtime Environment (build 1.8.0_162-8u162-b12-1-b12)
OpenJDK 64-Bit Server VM (build 25.162-b12, mixed mode) =~ [java|openjdk][[:space:]]version[[:space:]]\"1\.([0-9][0-9]*)\.?+ ]]
+ eval JAVA_MAJOR=8
++ JAVA_MAJOR=8
+ '[' 8 -lt 8 ']'
+ verify_java_home
+ '[' -z /usr/lib/jvm/java-1.8.0-openjdk-amd64 ']'
+ echo JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
+ . /opt/cloudera/cm-agent/service/common/cdh-default-hadoop
++ [[ -z 6 ]]
++ '[' 6 = 3 ']'
++ '[' 6 = -3 ']'
++ '[' 6 -ge 4 ']'
++ export HADOOP_HOME_WARN_SUPPRESS=true
++ HADOOP_HOME_WARN_SUPPRESS=true
++ export HADOOP_PREFIX=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ HADOOP_PREFIX=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ export HADOOP_LIBEXEC_DIR=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/libexec
++ HADOOP_LIBEXEC_DIR=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop/libexec
++ export HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format
++ HADOOP_CONF_DIR=/var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format
++ export HADOOP_COMMON_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ HADOOP_COMMON_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop
++ export HADOOP_HDFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs
++ HADOOP_HDFS_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs
++ export HADOOP_MAPRED_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce
++ HADOOP_MAPRED_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-mapreduce
++ '[' 6 = 4 ']'
++ '[' 6 -ge 5 ']'
++ export HADOOP_YARN_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn
++ HADOOP_YARN_HOME=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-yarn
++ replace_pid -Xms1961885696 -Xmx1961885696 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/hdfs_hdfs-NAMENODE-04c87db2a470ce2ac21ae3a8547c2609_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh
++ sed 's#{{PID}}#2811#g'
++ echo -Xms1961885696 -Xmx1961885696 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/hdfs_hdfs-NAMENODE-04c87db2a470ce2ac21ae3a8547c2609_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh
+ export 'HADOOP_NAMENODE_OPTS=-Xms1961885696 -Xmx1961885696 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/hdfs_hdfs-NAMENODE-04c87db2a470ce2ac21ae3a8547c2609_pid2811.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh'
+ HADOOP_NAMENODE_OPTS='-Xms1961885696 -Xmx1961885696 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/hdfs_hdfs-NAMENODE-04c87db2a470ce2ac21ae3a8547c2609_pid2811.hprof -XX:OnOutOfMemoryError=/opt/cloudera/cm-agent/service/common/killparent.sh'
++ replace_pid
++ echo
++ sed 's#{{PID}}#2811#g'
+ export HADOOP_DATANODE_OPTS=
+ HADOOP_DATANODE_OPTS=
++ replace_pid
++ echo
++ sed 's#{{PID}}#2811#g'
+ export HADOOP_SECONDARYNAMENODE_OPTS=
+ HADOOP_SECONDARYNAMENODE_OPTS=
++ replace_pid
++ echo
++ sed 's#{{PID}}#2811#g'
+ export HADOOP_NFS3_OPTS=
+ HADOOP_NFS3_OPTS=
++ replace_pid
++ echo
++ sed 's#{{PID}}#2811#g'
+ export HADOOP_JOURNALNODE_OPTS=
+ HADOOP_JOURNALNODE_OPTS=
+ '[' 6 -ge 4 ']'
+ HDFS_BIN=/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs/bin/hdfs
+ export 'HADOOP_OPTS=-Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Djava.net.preferIPv4Stack=true '
+ echo 'using /usr/lib/jvm/java-1.8.0-openjdk-amd64 as JAVA_HOME'
+ echo 'using 6 as CDH_VERSION'
+ echo 'using /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format as CONF_DIR'
+ echo 'using as SECURE_USER'
+ echo 'using as SECURE_GROUP'
+ set_hadoop_classpath
+ set_classpath_in_var HADOOP_CLASSPATH
+ '[' -z HADOOP_CLASSPATH ']'
+ [[ -n /opt/cloudera/cm ]]
++ find /opt/cloudera/cm/lib/plugins -maxdepth 1 -name '*.jar'
++ tr '\n' :
+ ADD_TO_CP=/opt/cloudera/cm/lib/plugins/tt-instrumentation-6.2.0.jar:/opt/cloudera/cm/lib/plugins/event-publish-6.2.0-shaded.jar:
+ [[ -n navigator/cdh6 ]]
+ for DIR in $CM_ADD_TO_CP_DIRS
++ find /opt/cloudera/cm/lib/plugins/navigator/cdh6 -maxdepth 1 -name '*.jar'
++ tr '\n' :
+ PLUGIN=/opt/cloudera/cm/lib/plugins/navigator/cdh6/audit-plugin-cdh6-6.2.0-shaded.jar:
+ ADD_TO_CP=/opt/cloudera/cm/lib/plugins/tt-instrumentation-6.2.0.jar:/opt/cloudera/cm/lib/plugins/event-publish-6.2.0-shaded.jar:/opt/cloudera/cm/lib/plugins/navigator/cdh6/audit-plugin-cdh6-6.2.0-shaded.jar:
+ eval 'OLD_VALUE=$HADOOP_CLASSPATH'
++ OLD_VALUE=
+ NEW_VALUE=/opt/cloudera/cm/lib/plugins/tt-instrumentation-6.2.0.jar:/opt/cloudera/cm/lib/plugins/event-publish-6.2.0-shaded.jar:/opt/cloudera/cm/lib/plugins/navigator/cdh6/audit-plugin-cdh6-6.2.0-shaded.jar:
+ export HADOOP_CLASSPATH=/opt/cloudera/cm/lib/plugins/tt-instrumentation-6.2.0.jar:/opt/cloudera/cm/lib/plugins/event-publish-6.2.0-shaded.jar:/opt/cloudera/cm/lib/plugins/navigator/cdh6/audit-plugin-cdh6-6.2.0-shaded.jar
+ HADOOP_CLASSPATH=/opt/cloudera/cm/lib/plugins/tt-instrumentation-6.2.0.jar:/opt/cloudera/cm/lib/plugins/event-publish-6.2.0-shaded.jar:/opt/cloudera/cm/lib/plugins/navigator/cdh6/audit-plugin-cdh6-6.2.0-shaded.jar
+ set -x
+ replace_conf_dir
+ echo CONF_DIR=/var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format
+ echo CMF_CONF_DIR=
+ EXCLUDE_CMF_FILES=('cloudera-config.sh' 'hue.sh' 'impala.sh' 'sqoop.sh' 'supervisor.conf' 'config.zip' 'proc.json' '*.log' '*.keytab' '*jceks')
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '*.log' hdfs.keytab '*jceks'
+ find /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format -type f '!' -path '/var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format/logs/*' '!' -name cloudera-config.sh '!' -name hue.sh '!' -name impala.sh '!' -name sqoop.sh '!' -name supervisor.conf '!' -name config.zip '!' -name proc.json '!' -name '*.log' '!' -name hdfs.keytab '!' -name '*jceks' -exec perl -pi -e 's#\{\{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format#g' '{}' ';'
Can't open /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format/supervisor_status: Permission denied.
+ make_scripts_executable
+ find /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format -regex '.*\.\(py\|sh\)$' -exec chmod u+x '{}' ';'
+ '[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']'
+ ulimit -l
+ export HADOOP_IDENT_STRING=hdfs
+ HADOOP_IDENT_STRING=hdfs
+ '[' -n '' ']'
+ '[' mkdir '!=' format-namenode ']'
+ acquire_kerberos_tgt hdfs.keytab
+ '[' -z hdfs.keytab ']'
+ KERBEROS_PRINCIPAL=
+ '[' '!' -z '' ']'
+ '[' -n '' ']'
+ '[' validate-writable-empty-dirs = format-namenode ']'
+ '[' file-operation = format-namenode ']'
+ '[' bootstrap = format-namenode ']'
+ '[' failover = format-namenode ']'
+ '[' transition-to-active = format-namenode ']'
+ '[' initializeSharedEdits = format-namenode ']'
+ '[' initialize-znode = format-namenode ']'
+ '[' format-namenode = format-namenode ']'
+ '[' -z /dfs/nn ']'
+ for dfsdir in $DFS_STORAGE_DIRS
+ '[' -e /dfs/nn ']'
+ '[' '!' -d /dfs/nn ']'
+ CLUSTER_ARGS=
+ '[' 2 -eq 2 ']'
+ CLUSTER_ARGS='-clusterId cluster19'
+ '[' 3 = 6 ']'
+ '[' -3 = 6 ']'
+ exec /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs/bin/hdfs --config /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format namenode -format -clusterId cluster19 -nonInteractive
WARNING: HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX.
WARNING: HADOOP_NAMENODE_OPTS has been replaced by HDFS_NAMENODE_OPTS. Using value of HADOOP_NAMENODE_OPTS.
Running in non-interactive mode, and data appears to exist in Storage Directory /dfs/nn. Not formatting.
... View more
07-18-2019
01:19 AM
This is the namenode log, and there is no any error STARTUP_MSG: build = http://github.com/cloudera/hadoop -r d1dff3d3a126da44e3458bbf148c3bc16ff55bd8; compiled by 'jenkins' on 2019-03-14T06:39Z
STARTUP_MSG: java = 1.8.0_162
************************************************************/
2019-07-18 15:52:05,190 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
2019-07-18 15:52:05,234 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: createNameNode [-format, -clusterId, cluster19, -nonInteractive]
2019-07-18 15:52:05,485 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Edit logging is async:true
2019-07-18 15:52:05,494 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: KeyProvider: null
2019-07-18 15:52:05,494 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsLock is fair: true
2019-07-18 15:52:05,495 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
2019-07-18 15:52:05,498 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner = hdfs (auth:SIMPLE)
2019-07-18 15:52:05,499 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup = supergroup
2019-07-18 15:52:05,499 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = true
2019-07-18 15:52:05,499 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false
2019-07-18 15:52:05,522 INFO org.apache.hadoop.hdfs.server.common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
2019-07-18 15:52:05,532 WARN org.apache.hadoop.hdfs.util.CombinedHostsFileReader: /var/run/cloudera-scm-agent/process/40-hdfs-NAMENODE-format/dfs_all_hosts.txt has invalid JSON format.Try the old format without top-level token defined.
2019-07-18 15:52:05,573 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
2019-07-18 15:52:05,573 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
2019-07-18 15:52:05,575 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
2019-07-18 15:52:05,576 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: The block deletion will start around 2019 Jul 18 15:52:05
2019-07-18 15:52:05,577 INFO org.apache.hadoop.util.GSet: Computing capacity for map BlocksMap
2019-07-18 15:52:05,577 INFO org.apache.hadoop.util.GSet: VM type = 64-bit
2019-07-18 15:52:05,578 INFO org.apache.hadoop.util.GSet: 2.0% max memory 1.8 GB = 36.2 MB
2019-07-18 15:52:05,579 INFO org.apache.hadoop.util.GSet: capacity = 2^22 = 4194304 entries
2019-07-18 15:52:05,598 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable = false
2019-07-18 15:52:05,601 INFO org.apache.hadoop.conf.Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
2019-07-18 15:52:05,601 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 1
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication = 3
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication = 512
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication = 1
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams = 20
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer = false
2019-07-18 15:52:05,602 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog = 1000
2019-07-18 15:52:05,618 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: GLOBAL serial map: bits=24 maxEntries=16777215
2019-07-18 15:52:05,627 INFO org.apache.hadoop.util.GSet: Computing capacity for map INodeMap
2019-07-18 15:52:05,627 INFO org.apache.hadoop.util.GSet: VM type = 64-bit
2019-07-18 15:52:05,627 INFO org.apache.hadoop.util.GSet: 1.0% max memory 1.8 GB = 18.1 MB
2019-07-18 15:52:05,627 INFO org.apache.hadoop.util.GSet: capacity = 2^21 = 2097152 entries
2019-07-18 15:52:05,630 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: ACLs enabled? false
2019-07-18 15:52:05,630 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: POSIX ACL inheritance enabled? true
2019-07-18 15:52:05,630 INFO org.apache.hadoop.hdfs.server.namenode.FSDirectory: XAttrs enabled? true
2019-07-18 15:52:05,630 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occurring more than 10 times
2019-07-18 15:52:05,634 INFO org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager: Loaded config captureOpenFiles: true, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true
2019-07-18 15:52:05,637 INFO org.apache.hadoop.util.GSet: Computing capacity for map cachedBlocks
2019-07-18 15:52:05,637 INFO org.apache.hadoop.util.GSet: VM type = 64-bit
2019-07-18 15:52:05,637 INFO org.apache.hadoop.util.GSet: 0.25% max memory 1.8 GB = 4.5 MB
2019-07-18 15:52:05,637 INFO org.apache.hadoop.util.GSet: capacity = 2^19 = 524288 entries
2019-07-18 15:52:05,643 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
2019-07-18 15:52:05,643 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
2019-07-18 15:52:05,643 INFO org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
2019-07-18 15:52:05,646 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache on namenode is enabled
2019-07-18 15:52:05,646 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
2019-07-18 15:52:05,648 INFO org.apache.hadoop.util.GSet: Computing capacity for map NameNodeRetryCache
2019-07-18 15:52:05,648 INFO org.apache.hadoop.util.GSet: VM type = 64-bit
2019-07-18 15:52:05,648 INFO org.apache.hadoop.util.GSet: 0.029999999329447746% max memory 1.8 GB = 555.9 KB
2019-07-18 15:52:05,648 INFO org.apache.hadoop.util.GSet: capacity = 2^16 = 65536 entries
2019-07-18 15:52:05,666 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1: ExitException
2019-07-18 15:52:05,667 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at device1/192.168.0.104
************************************************************/
... View more
07-17-2019
10:33 PM
Thanks Eric , I'll take a shot for that.
... View more
07-17-2019
10:22 PM
I wanna add a new server to cluster. But there was an issue in the last link. My namenode is device1, and I was going to add device2 to the cluster. The information printed as below: Failed to perform First Run of services.
Completed 1 of 1 step(s).
Show All Steps Show Only Failed Steps Show Only Running Steps
Run a set of services for the first time
Completed only 4/5 steps. First failure: Command (Format (102)) has failed
Jul 18, 12:52:45 PM 22.51s
Execute 7 steps in sequence
Completed only 4/5 steps. First failure: Command (Format (102)) has failed
Jul 18, 12:52:45 PM 22.45s
Execute 5 steps in parallel
Completed only 4/5 steps. First failure: Command (Format (102)) has failed
Jul 18, 12:52:50 PM 17.34s
Formatting the name directories of the current NameNode. If the name directories are not empty, this is expected to fail.
Command (Format (102)) has failed
NameNode (device1)
Jul 18, 12:52:50 PM 17.34s
Format
Failed to format NameNode.
NameNode (device1)
Jul 18, 12:52:50 PM 17.33s
$> hdfs/hdfs.sh ["format-namenode","cluster19"]
stdout
stderr
Role Log
Can't open /var/run/cloudera-scm-agent/process/32-hdfs-NAMENODE-format/supervisor_status: Permission denied.
+ make_scripts_executable
+ find /var/run/cloudera-scm-agent/process/32-hdfs-NAMENODE-format -regex '.*\.\(py\|sh\)$' -exec chmod u+x '{}' ';'
+ '[' DATANODE_MAX_LOCKED_MEMORY '!=' '' ']'
+ ulimit -l
+ export HADOOP_IDENT_STRING=hdfs
+ HADOOP_IDENT_STRING=hdfs
+ '[' -n '' ']'
+ '[' mkdir '!=' format-namenode ']'
+ acquire_kerberos_tgt hdfs.keytab
+ '[' -z hdfs.keytab ']'
+ KERBEROS_PRINCIPAL=
+ '[' '!' -z '' ']'
+ '[' -n '' ']'
+ '[' validate-writable-empty-dirs = format-namenode ']'
+ '[' file-operation = format-namenode ']'
+ '[' bootstrap = format-namenode ']'
+ '[' failover = format-namenode ']'
+ '[' transition-to-active = format-namenode ']'
+ '[' initializeSharedEdits = format-namenode ']'
+ '[' initialize-znode = format-namenode ']'
+ '[' format-namenode = format-namenode ']'
+ '[' -z /dfs/nn ']'
+ for dfsdir in $DFS_STORAGE_DIRS
+ '[' -e /dfs/nn ']'
+ '[' '!' -d /dfs/nn ']'
+ CLUSTER_ARGS=
+ '[' 2 -eq 2 ']'
+ CLUSTER_ARGS='-clusterId cluster19'
+ '[' 3 = 6 ']'
+ '[' -3 = 6 ']'
+ exec /opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/lib/hadoop-hdfs/bin/hdfs --config /var/run/cloudera-scm-agent/process/32-hdfs-NAMENODE-format namenode -format -clusterId cluster19 -nonInteractive
WARNING: HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX.
WARNING: HADOOP_NAMENODE_OPTS has been replaced by HDFS_NAMENODE_OPTS. Using value of HADOOP_NAMENODE_OPTS.
Running in non-interactive mode, and data appears to exist in Storage Directory /dfs/nn. Not formatting. screenshot
... View more
Labels:
- Labels:
-
Cloudera Manager
07-16-2019
10:38 PM
Thanks Michalis, just checked the db.properties as below: -rw-r--r-- 1 cloudera-scm cloudera-scm 757 3月 15 01:45 /etc/cloudera-scm-server/db.properties.~1~
-rw------- 1 cloudera-scm cloudera-scm 446 6月 18 19:37 /etc/cloudera-scm-server/db.properties.~2~
-rw------- 1 cloudera-scm cloudera-scm 448 6月 18 19:39 /etc/cloudera-scm-server/db.properties.~3~
-rw------- 1 cloudera-scm cloudera-scm 448 6月 18 19:39 /etc/cloudera-scm-server/db.properties.~4~
-rw------- 1 cloudera-scm cloudera-scm 446 6月 18 19:40 /etc/cloudera-scm-server/db.properties.~5~
-rw------- 1 cloudera-scm cloudera-scm 448 6月 18 19:40 /etc/cloudera-scm-server/db.properties.~6~
-rw------- 1 cloudera-scm cloudera-scm 443 6月 18 19:41 /etc/cloudera-scm-server/db.properties.~7~
-rw------- 1 cloudera-scm cloudera-scm 447 6月 18 19:41 /etc/cloudera-scm-server/db.properties.~8~
-rw------- 1 cloudera-scm cloudera-scm 441 6月 18 19:41 /etc/cloudera-scm-server/db.properties.~9~
-rw------- 1 cloudera-scm cloudera-scm 446 6月 18 19:41 /etc/cloudera-scm-server/db.properties.~10~
-rw------- 1 cloudera-scm cloudera-scm 452 6月 18 19:42 /etc/cloudera-scm-server/db.properties.~11~
-rw------- 1 cloudera-scm cloudera-scm 450 6月 18 19:42 /etc/cloudera-scm-server/db.properties.~12~
-rw------- 1 cloudera-scm cloudera-scm 450 6月 18 19:42 /etc/cloudera-scm-server/db.properties.~13~
-rw------- 1 cloudera-scm cloudera-scm 453 6月 18 19:43 /etc/cloudera-scm-server/db.properties.~14~
-rw------- 1 cloudera-scm cloudera-scm 446 6月 18 21:20 /etc/cloudera-scm-server/db.properties.~15~
-rw------- 1 cloudera-scm cloudera-scm 453 6月 19 13:05 /etc/cloudera-scm-server/db.properties.~16~
-rw------- 1 cloudera-scm cloudera-scm 442 7月 16 14:34 /etc/cloudera-scm-server/db.properties.~17~
-rw------- 1 cloudera-scm cloudera-scm 446 7月 16 14:34 /etc/cloudera-scm-server/db.properties.~18~
-rw------- 1 cloudera-scm cloudera-scm 458 7月 16 14:37 /etc/cloudera-scm-server/db.properties.~19~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 14:37 /etc/cloudera-scm-server/db.properties.~20~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 15:02 /etc/cloudera-scm-server/db.properties.~21~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 15:03 /etc/cloudera-scm-server/db.properties.~22~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 15:04 /etc/cloudera-scm-server/db.properties.~23~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 16:46 /etc/cloudera-scm-server/db.properties.~24~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 16:47 /etc/cloudera-scm-server/db.properties.~25~
-rw------- 1 cloudera-scm cloudera-scm 454 7月 16 16:47 /etc/cloudera-scm-server/db.properties.~26~
-rw------- 1 cloudera-scm cloudera-scm 453 7月 16 16:47 /etc/cloudera-scm-server/db.properties.~27~
-rw------- 1 cloudera-scm cloudera-scm 446 7月 16 17:03 /etc/cloudera-scm-server/db.properties.~28~
-rw------- 1 cloudera-scm cloudera-scm 446 7月 16 17:03 /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.setupType=EXTERNAL
com.cloudera.cmf.db.password=mypassword few days ago my CM started successfully. But, after restarted my server, it raised that error when I started. Now I tried to drop all the databases in mysql and create again, then run the following command respectively : sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql amon amon mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql rman rman mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql hue hue mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql metastore hive mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql sentry sentry mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql nav nav mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql navms navms mypassword sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql oozie oozie mypassword
... View more
07-16-2019
10:13 PM
Hi Eric , my db.properties is as this: com.cloudera.cmf.db.type=mysql
com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.setupType=EXTERNAL
com.cloudera.cmf.db.password=mypassword And I just used the command to configure: sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql metastore hive mypassword It started successfully in the beginning, but it occured that error few days later after I restarted my server. now I dropped the database scm and try starting again.
... View more
07-16-2019
01:59 AM
I 'm using CDH-6.2.0-1.cdh6.2.0. and I use mysql as metastore
1. I have already configured the mysql meta first by running: sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql metastore hive
2. I use this command to start:
sudo systemctl start cloudera-scm-server.
Then it tells me as below:
2019-07-16 15:06:38,947 ERROR main:com.cloudera.server.cmf.Main: Server failed.
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.cloudera.server.cmf.TrialState': Cannot resolve reference to bean 'entityManagerFactoryBean' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.RuntimeException: Unable to obtain CM release version.
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:634)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
at com.cloudera.server.cmf.Main.bootstrapSpringContext(Main.java:418)
at com.cloudera.server.cmf.Main.<init>(Main.java:260)
at com.cloudera.server.cmf.Main.main(Main.java:233)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.RuntimeException: Unable to obtain CM release version.
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
... 17 more
Caused by: java.lang.RuntimeException: Unable to obtain CM release version.
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getPersistedCmRelease(EntityManagerFactoryBean.java:311)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.checkVersionDoFail(EntityManagerFactoryBean.java:358)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:172)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:75)
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
... 22 more
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:155)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1407)
at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1444)
at com.cloudera.cmf.persist.DbVersionDao.getVersion(DbVersionDao.java:53)
at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getPersistedCmRelease(EntityManagerFactoryBean.java:307)
... 26 more
Caused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:63)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2123)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1911)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1887)
at org.hibernate.loader.Loader.doQuery(Loader.java:932)
at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349)
at org.hibernate.loader.Loader.doList(Loader.java:2615)
at org.hibernate.loader.Loader.doList(Loader.java:2598)
at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430)
at org.hibernate.loader.Loader.list(Loader.java:2425)
at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:335)
at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:2128)
at org.hibernate.internal.AbstractSharedSessionContract.list(AbstractSharedSessionContract.java:991)
at org.hibernate.query.internal.NativeQueryImpl.doList(NativeQueryImpl.java:147)
at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1398)
... 29 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'metastore.CM_VERSION' doesn't exist
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.Util.getInstance(Util.java:408)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:944)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858)
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1966)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:116)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70)
Mysql database is as below:
+---------------------------+
| Tables_in_metastore |
+---------------------------+
| AUDITS |
| BUCKETING_COLS |
| CDH_VERSION |
| CDS |
| COLUMNS_V2 |
| COMMANDS |
| CONFIGS |
| CREDENTIALS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HOSTS |
| IDXS |
| INDEX_PARAMS |
| KEY_CONSTRAINTS |
| MASTER_KEYS |
| NOTIFICATION_LOG |
| NOTIFICATION_SEQUENCE |
| NUCLEUS_TABLES |
| PARTITIONS |
| PARTITION_EVENTS |
| PARTITION_KEYS |
| PARTITION_KEY_VALS |
| PARTITION_PARAMS |
| PART_COL_PRIVS |
| PART_COL_STATS |
| PART_PRIVS |
| PROCESSES |
| ROLES |
| ROLE_MAP |
| SCHEMA_VERSION |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SKEWED_COL_NAMES |
| SKEWED_COL_VALUE_LOC_MAP |
| SKEWED_STRING_LIST |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES |
| SORT_COLS |
| TABLE_PARAMS |
| TAB_COL_STATS |
| TBLS |
| TBL_COL_PRIVS |
| TBL_PRIVS |
| TYPES |
| TYPE_FIELDS |
| VERSION |
... View more
Labels:
- Labels:
-
Cloudera Manager
-
Hive
07-16-2019
12:30 AM
I use this command to start: sudo systemctl start cloudera-scm-server. I 'm using CDH-6.2.0-1.cdh6.2.0. and I use mysql as metastore. Then it tells me as below: ``` 2019-07-16 15:06:38,947 ERROR main:com.cloudera.server.cmf.Main: Server failed. org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.cloudera.server.cmf.TrialState': Cannot resolve reference to bean 'entityManagerFactoryBean' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.RuntimeException: Unable to obtain CM release version. at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108) at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:634) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:145) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1198) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1100) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:511) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) at com.cloudera.server.cmf.Main.bootstrapSpringContext(Main.java:418) at com.cloudera.server.cmf.Main.<init>(Main.java:260) at com.cloudera.server.cmf.Main.main(Main.java:233) Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is java.lang.RuntimeException: Unable to obtain CM release version. at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:185) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1640) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:254) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) ... 17 more Caused by: java.lang.RuntimeException: Unable to obtain CM release version. at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getPersistedCmRelease(EntityManagerFactoryBean.java:311) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.checkVersionDoFail(EntityManagerFactoryBean.java:358) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:172) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:75) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178) ... 22 more Caused by: javax.persistence.PersistenceException: org.hibernate.exception.SQLGrammarException: could not extract ResultSet at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147) at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:155) at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1407) at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1444) at com.cloudera.cmf.persist.DbVersionDao.getVersion(DbVersionDao.java:53) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getPersistedCmRelease(EntityManagerFactoryBean.java:307) ... 26 more Caused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSet at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:63) at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:79) at org.hibernate.loader.Loader.getResultSet(Loader.java:2123) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1911) at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1887) at org.hibernate.loader.Loader.doQuery(Loader.java:932) at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:349) at org.hibernate.loader.Loader.doList(Loader.java:2615) at org.hibernate.loader.Loader.doList(Loader.java:2598) at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2430) at org.hibernate.loader.Loader.list(Loader.java:2425) at org.hibernate.loader.custom.CustomLoader.list(CustomLoader.java:335) at org.hibernate.internal.SessionImpl.listCustomQuery(SessionImpl.java:2128) at org.hibernate.internal.AbstractSharedSessionContract.list(AbstractSharedSessionContract.java:991) at org.hibernate.query.internal.NativeQueryImpl.doList(NativeQueryImpl.java:147) at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1398) ... 29 more Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'metastore.CM_VERSION' doesn't exist at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.mysql.jdbc.Util.handleNewInstance(Util.java:425) at com.mysql.jdbc.Util.getInstance(Util.java:408) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:944) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1858) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1966) at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:116) at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70) ``` Mysql database is as below: ``` +---------------------------+ | Tables_in_metastore | +---------------------------+ | AUDITS | | BUCKETING_COLS | | CDH_VERSION | | CDS | | COLUMNS_V2 | | COMMANDS | | CONFIGS | | CREDENTIALS | | DATABASE_PARAMS | | DBS | | DB_PRIVS | | DELEGATION_TOKENS | | FUNCS | | FUNC_RU | | GLOBAL_PRIVS | | HOSTS | | IDXS | | INDEX_PARAMS | | KEY_CONSTRAINTS | | MASTER_KEYS | | NOTIFICATION_LOG | | NOTIFICATION_SEQUENCE | | NUCLEUS_TABLES | | PARTITIONS | | PARTITION_EVENTS | | PARTITION_KEYS | | PARTITION_KEY_VALS | | PARTITION_PARAMS | | PART_COL_PRIVS | | PART_COL_STATS | | PART_PRIVS | | PROCESSES | | ROLES | | ROLE_MAP | | SCHEMA_VERSION | | SDS | | SD_PARAMS | | SEQUENCE_TABLE | | SERDES | | SERDE_PARAMS | | SKEWED_COL_NAMES | | SKEWED_COL_VALUE_LOC_MAP | | SKEWED_STRING_LIST | | SKEWED_STRING_LIST_VALUES | | SKEWED_VALUES | | SORT_COLS | | TABLE_PARAMS | | TAB_COL_STATS | | TBLS | | TBL_COL_PRIVS | | TBL_PRIVS | | TYPES | | TYPE_FIELDS | | VERSION | ```
... View more
Labels:
- Labels:
-
Cloudera Manager