pankaj_pankasin
Explorer
My Accepted Solutions
Show All

Re: HDP 2.6.5 fresh installation stuck. ranger pro...

by Explorer in Support Questions
‎09-04-2018 07:21 AM
‎09-04-2018 07:21 AM
what do u mean by running same environment? you mean ranger and ranger kms? database is running in google cloud? ... View more

HDP 2.6.5 fresh installation stuck. ranger process...

by Explorer in Support Questions
‎08-30-2018 05:59 AM
‎08-30-2018 05:59 AM
I am trying to install HDP 2.6.5 through cloud break in google cloud. I am able to install all the services in google cloud but ranger. Error stack trace is listed below. I tried to follow the posted solutions earlier but none of them is helping including java patch solution by deleting an entry from the table. Can someone help me to resolve this issue? 2018-08-30 05:47:57,312 [I] --------- Verifying Ranger DB connection --------- 2018-08-30 05:47:57,312 [I] Checking connection 2018-08-30 05:47:57,312 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-9.1-901-1.jdbc4.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://xx.xxx.xxx.xxx/ranger -u postgres -p '********' -noheader -trim -c \; -query "SELECT 1;" 2018-08-30 05:47:59,027 [I] connection success 2018-08-30 05:47:59,027 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-9.1-901-1.jdbc4.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://xx.xxx.xxx.xxx/ranger -u postgres -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by = 'Ranger 0.7.0.2.6.5.0-292' and active = 'Y';" 2018-08-30 05:48:00,758 [I] ----------------- Applying java patches ------------ 2018-08-30 05:48:00,759 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-9.1-901-1.jdbc4.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://xx.xxx.xxx.xxx/ranger -u postgres -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10001' and active = 'Y';" 2018-08-30 05:48:02,430 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-9.1-901-1.jdbc4.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://xx.xxx.xxx.xxx/ranger -u postgres -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'J10001' and active = 'N';" 2018-08-30 05:48:04,113 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/hdp/current/ranger-admin/ews/lib/postgresql-9.1-901-1.jdbc4.jar:/usr/hdp/current/ranger-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://xx.xxx.xxx.xxx/ranger -u postgres -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('J10001', current_timestamp, 'Ranger 0.7.0.2.6.5.0-292', current_timestamp, 'hdpcluster-m-1-20180830031406.c.striped-orbit-213714.internal','N') ;" 2018-08-30 05:48:05,791 [I] java patch PatchPasswordEncryption_J10001 is being applied. ... View more

Re: Cloud break Hortonworks deployment into cloud....

by Explorer in Support Questions
‎08-28-2018 08:34 AM
‎08-28-2018 08:34 AM
I looked at this document. i guess it's for on-premise servers. What about cloud? will cloud break support it. Latest cloud break supports HDP 2.6 and HDF 3.1. ... View more

Restarting the hortonworks cluster in cloud

by Explorer in Support Questions
‎08-28-2018 07:54 AM
‎08-28-2018 07:54 AM
I am running services in google cloud through hortonworks cloudbreak. Whats the proper way to restart the services. I could see stop option in cloud break but don't see any restart/start option ... View more
Labels:

Re: Cloud break Hortonworks deployment into cloud....

by Explorer in Support Questions
‎08-23-2018 09:39 AM
‎08-23-2018 09:39 AM
Is their anyway to deploy HDP & HDF together in cloud? ... View more

How to backup the data for hortonworks instances (...

by Explorer in Support Questions
‎08-21-2018 02:10 PM
‎08-21-2018 02:10 PM
We intent to build datalake in cloud. Can some one help me to understand how the data is backed up in cloud for Hive, Hbase, Hadoop (entire stack)?.. In a way how to make sure nodes are never gets corrupted all the time? Looking for some best practices for building datalake in cloud. For ex: For feed or any file drop we can create a S3 bucket in AWS.. Does similar principles also applied for Hadoop, HBase, Hive (I guess no..) as they are distributed and data will be replicated.. (+) How to deploy the Hadoop instances in multi region (i.e. if entire data center goes down).. ... View more

Re: Hortonworks Hadoop platform vs datalake (looki...

by Explorer in Support Questions
‎08-19-2018 01:51 AM
‎08-19-2018 01:51 AM
Thanks for your answer. Our dataset is not huge may be 10-15 TB of data at the moment. or over the period of 2-3 yrs grow to 15-20 TB at max. We intent to build this datalake in cloud but not using cloud provider platform as they don't provide data gov and lineage tracking.. our data set is mostly going to be unstructured.. ... View more

Re: Hortonworks Hadoop platform vs datalake (looki...

by Explorer in Support Questions
‎08-19-2018 01:47 AM
‎08-19-2018 01:47 AM
I assume you are referring to multi-data lake in your organization? or you are referring to processing the same feed in same datalake based on different kind of requirement? Why not storing the data in same datalake? as you mentioned we got multiple production environments.. ... View more

Cloud break Hortonworks deployment into cloud... (...

by Explorer in Support Questions
‎08-18-2018 01:51 PM
‎08-18-2018 01:51 PM
I would like to know how to upgrade the HDP cluster into cloud deployed through cloud break? Additionally, is it possible to install HDF AND HDP together into cloud if yes how? ... View more

Hortonworks Hadoop platform vs datalake (looking f...

by Explorer in Support Questions
‎08-18-2018 01:48 PM
‎08-18-2018 01:48 PM
have any one build a datalake in organisations? Looking for some best practices around it.. Should we have one single datalake or multi data lake within an organisations.. Looking for some guidance around it.. appreciate if some one can answer or point in right direction. I have googled out many links but unable to find one answers so looking for some real live examples.. ... View more
Labels:

HDFS Upgrade finalized state. HDFS cluster is not ...

by Explorer in Support Questions
‎08-14-2018 04:13 AM
‎08-14-2018 04:13 AM
Currently i am facing an issue while upgrading hortonworks stack (from 2.6.5 to 3.0.0). I get an error HDFS cluster is not finalised yet. screen-shot-2018-08-14-at-121041-pm.png.. HDP upgrade process is stuck as well. Dismiss button is not functional too. Can some one recommend what to look for? ... View more

Re: hortonworks upgrade process stucks. how to can...

by Explorer in Support Questions
‎08-14-2018 04:01 AM
‎08-14-2018 04:01 AM
screen-shot-2018-08-14-at-120123-pm.pngscreen-shot-2018-08-14-at-120111-pm.pngscreen-shot-2018-08-14-at-120055-pm.png PFA screenshot of process. ... View more

hortonworks upgrade process stucks. how to cancel ...

by Explorer in Support Questions
‎08-14-2018 03:59 AM
‎08-14-2018 03:59 AM
I am upgrading HDP stack and currently upgrade is stuck. Dismiss button is not work too. There was an error in installing the hive, it was fixed and hive was restarted from command line. As the process is stuck.. resuming option also not working. what to do next can some1 recommend me..? ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 03:53 AM
‎08-14-2018 03:53 AM
I restarted hive metastore by running tool, post command line start (https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_command-line-installation/content/validate_installation.html).. but ambari admin ui is still stuck.. is their a way we can restart upgrade or rollback or do something.. thr is no button to restart upgrade even..screen-shot-2018-08-14-at-115110-am.png ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 03:52 AM
‎08-14-2018 03:52 AM
screen-shot-2018-08-14-at-115110-am.png I restarted hive metastore by running tool, post command line start (https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_command-line-installation/content/validate_installation.html).. but ambari admin ui is still stuck.. is their a way we can restart upgrade or rollback or do something.. thr is no button to restart upgrade even.. ... View more

Re: hortonworks upgrade failed hive metastore sche...

by Explorer in Support Questions
‎08-14-2018 02:55 AM
‎08-14-2018 02:55 AM
I see a similar thread being discussed on MAPR.. https://community.mapr.com/community/exchange/blog/2017/06/19/handling-hive-metastore-db-table-version-inconsistency-when-hive-client-uses-embedded-metastore ... View more

hortonworks upgrade failed hive metastore schema d...

by Explorer in Support Questions
‎08-14-2018 02:33 AM
‎08-14-2018 02:33 AM
Stack trace from hive meta store log: Is there a way to fix it or resolve it? p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} p.p2 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff; min-height: 13.0px} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre} 2018-08-14T10:26:53,488 INFO[main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(186)) - Using direct SQL, underlying DB is MYSQL 2018-08-14T10:26:53,490 INFO[main]: metastore.ObjectStore (ObjectStore.java:setConf(396)) - Initialized ObjectStore 2018-08-14T10:26:55,362 ERROR [main]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(201)) - MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9063) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) at com.sun.proxy.$Proxy32.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:691) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:539) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8799) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8794) at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:9064) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8981) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) 2018-08-14T10:26:55,364 ERROR [main]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:<init>(83)) - HMSHandler Fatal error: MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9063) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) at com.sun.proxy.$Proxy32.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:691) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:539) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8799) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8794) at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:9064) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8981) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) 2018-08-14T10:26:55,364 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(9183)) - MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8799) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8794) at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:9064) at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8981) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:318) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) Caused by: MetaException(message:Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt) at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9063) at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) at com.sun.proxy.$Proxy32.verifySchema(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:691) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:539) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) ... 11 more 2018-08-14T10:26:55,365 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(8986)) - Metastore Thrift Server threw an exception... org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8799) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8794) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:9064) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8981) [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.0.3.0.0.0-1634.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.0.3.0.0.0-1634.jar:?] Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9063) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at com.sun.proxy.$Proxy32.verifySchema(Unknown Source) ~[?:?] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:691) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:539) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] ... 11 more 2018-08-14T10:26:55,379 INFO[pool-2-thread-1]: metastore.HiveMetaStore (HiveMetaStore.java:lambda$main$0(8951)) - Shutting down hive metastore. 2018-08-14T10:26:55,393 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(210)) - Stopping hivemetastore metrics system... 2018-08-14T10:26:55,393 INFO[timeline]: impl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(141)) - timeline thread interrupted. 2018-08-14T10:26:55,394 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(216)) - hivemetastore metrics system stopped. 2018-08-14T10:26:55,394 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(607)) - hivemetastore metrics system shutdown complete. 2018-08-14T10:26:55,395 INFO[pool-2-thread-1]: metastore.HiveMetaStore (HiveMetaStore.java:lambda$startupShutdownMessage$1(9389)) - SHUTDOWN_MSG: ... View more
Labels:

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 02:28 AM
‎08-14-2018 02:28 AM
on further debugging i could see this issue in hive metastore log: p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Menlo; color: #000000; background-color: #ffffff} span.s1 {font-variant-ligatures: no-common-ligatures} span.Apple-tab-span {white-space:pre} 018-08-14T10:26:55,365 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(8986)) - Metastore Thrift Server threw an exception... org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:84) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8799) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:8794) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:9064) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8981) [hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.0.3.0.0.0-1634.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.0.3.0.0.0-1634.jar:?] Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Hive Schema version 3.1.0 does not match metastore's schema version 2.1.2000 Metastore is not upgraded or corrupt at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:9063) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:9027) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at com.sun.proxy.$Proxy32.verifySchema(Unknown Source) ~[?:?] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:698) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:691) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:768) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:539) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_162] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_162] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_162] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_162] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80) ~[hive-exec-3.1.0.3.0.0.0-1634.jar:3.1.0.3.0.0.0-1634] ... 11 more 2018-08-14T10:26:55,379 INFO[pool-2-thread-1]: metastore.HiveMetaStore (HiveMetaStore.java:lambda$main$0(8951)) - Shutting down hive metastore. 2018-08-14T10:26:55,393 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(210)) - Stopping hivemetastore metrics system... 2018-08-14T10:26:55,393 INFO[timeline]: impl.MetricsSinkAdapter (MetricsSinkAdapter.java:publishMetricsFromQueue(141)) - timeline thread interrupted. 2018-08-14T10:26:55,394 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(216)) - hivemetastore metrics system stopped. 2018-08-14T10:26:55,394 INFO[pool-2-thread-1]: impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(607)) - hivemetastore metrics system shutdown complete. 2018-08-14T10:26:55,395 INFO[pool-2-thread-1]: metastore.HiveMetaStore (HiveMetaStore.java:lambda$startupShutdownMessage$1(9389)) - SHUTDOWN_MSG: ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 02:17 AM
‎08-14-2018 02:17 AM
screen-shot-2018-08-14-at-101741-am.png uploading few more pictures for guidance how to troubleshoot.. ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 02:14 AM
‎08-14-2018 02:14 AM
PFA screenshot from the admin page UI. Funny part is DISMISS button is not even working. it's just stuck. Is there something can be tried out from the back end? How and where to check logs can u guide me ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 02:13 AM
‎08-14-2018 02:13 AM
PFA screenshot from the admin page UI. Funny part is DISMISS button is not even working. it's just stuck. Is there something can be tried out from the back end? How and where to check logs. can you guide me? ... View more

Re: hortonworks HDP 3.0 update stuck in progress f...

by Explorer in Support Questions
‎08-14-2018 02:13 AM
‎08-14-2018 02:13 AM
screen-shot-2018-08-14-at-101042-am.pngscreen-shot-2018-08-14-at-101055-am.pngscreen-shot-2018-08-14-at-101107-am.png PFA screenshot from the admin page UI. Funny part is DISMISS button is not even working. it's just stuck. Is there something can be tried out from the back end? ... View more

hortonworks HDP 3.0 update stuck in progress for m...

by Explorer in Support Questions
‎08-13-2018 04:17 AM
‎08-13-2018 04:17 AM
Urgently looking for help. We tried upgrading hortonworks HDP from 2.6.5 to 3.0.0 and it's tuck since last 4-5 days. I am new to hortonworks working since last 7-8 months.. Can someone recommend how to fix it up and where to look for.PFA for reference.. ..screen-shot-2018-08-13-at-121206-pm.png ... View more

HDP platform Apache atlas configuration with Confl...

by Explorer in Support Questions
‎08-02-2018 01:38 PM
‎08-02-2018 01:38 PM
Is it possible to configure Confluent Kafka with HDP platform? As HDP platform only supports version 0.10.X wondering if they have plan to support upgraded version of KAFKA to use Kafka connect and KAFKA stream API. i would like to use Kafka connect and Kafka stream API for our project at the same time want to leverage HDP platform offerings.. I noticed kafka connect scripts in kafka shipped with HDP platform but they left the port default to 9092 while kafka port was changed to 6067... though the port could be changed but wondering if they support all feature of kafka-connect. For now our concern is only lineage tracking of confluent kafka.. ... View more
Labels:

Re: How to setup kafka cluster on hortonworks plat...

by Explorer in Support Questions
‎07-02-2018 07:44 AM
‎07-02-2018 07:44 AM
I apology for not putting the answer earlier. I was able to resolve the issue. Issue was due to ranger policy setup on Kafka topics. After turning off the ranger policy it works out of the box. Kafka cluster creates a meta.properties folder. HDP infra setups the Kafka in cluster from start ... View more

Re: Need help in understanding the PII tag in ATLA...

by Explorer in Support Questions
‎07-01-2018 07:58 AM
‎07-01-2018 07:58 AM
Thanks!. After troubleshooting i realized it was an issue due to setup of kafka ranger plugin.. ATLAS and RANGER were unable to communicate. ... View more

How to setup kafka cluster on hortonworks platform

by Explorer in Support Questions
‎06-30-2018 02:49 PM
‎06-30-2018 02:49 PM
Can some one point me in right direction of how to setup KAFKA cluster on HDP platform. To setup kafka clusters usually we add broker.id=1,2,3... but hortonworks doesn't provide this option to add in thr.. Issue: is when i try to create kafka topics i am able to create with 2,3 partitions and replicas 2,3.. but starting consumer and producer start throwing exception unknown topic or partition. My guess is it;s due to kafka HA configuration.. ... View more
Labels:

tagsync how does it work? is it dependent on kafka

by Explorer in Support Questions
‎06-24-2018 04:11 PM
‎06-24-2018 04:11 PM
Can some one help me to understand what is tag sync and how does it work? I don't see any log being created in tag sync folder under /var/log/..tag/sync.. ATLAS tags are not showing up on RANGER UI. ... View more
Labels:

Re: Need help in understanding the PII tag in ATLA...

by Explorer in Support Questions
‎06-24-2018 03:02 PM
‎06-24-2018 03:02 PM
For me it doesn't work at 2nd step creating a new policy because both the plocies ar contradictory to each other, ... View more

Need help in understanding the PII tag in ATLAS & ...

by Explorer in Support Questions
‎06-24-2018 03:01 PM
‎06-24-2018 03:01 PM
Can some one explain me how PII tag actually works? I am following this link to create working example over here https://hortonworks.com/tutorial/tag-based-policies-with-apache-ranger-and-apache-atlas/#step-6-create-a-ranger-policy-to-limit-access-of-hive-data My doubt is: 1. At first, user raj_ops and maria_dev are given permission to table "Figure 15: Sandbox_hive Repository’s Policies" which allow them to query the record successfully. 2. As per "Figure 26: Add New Ranger Policy" is created a new policy to exlcude PII column. I believe at this point both the policies are contradictory to each other (25, 28/26). user are allowed to select only name column. 3. Later as per PII tag further down the tag use can query the data. Can some one tell me or point me step be step working example of PII tag. or any tag based policies. ... View more
Contact Me
Online
Offline
Last Visited
‎01-17-2019 09:53 AM
Public Statistics
Date Registered ‎05-15-2018 03:02 PM
Last Visited ‎01-17-2019 09:53 AM
Total Messages Posted 42