Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
nehanirmal
nehanirmal
Explorer
My Accepted Solutions
Show All

Hortonworks services donot come up on their own af...

by Explorer nehanirmal in Support Questions
‎01-10-2019 01:52 PM
‎01-10-2019 01:52 PM
Hi Team, After patching all my services are unable to come up on their own. I manually follow these steps to bring it up else get a namenode checkpoint error: 1. Start the Zookeeper First 2. Goto HDFS and start the Journal Nodes 3. Start both the zkfailover controllers in hdfs 4. Open Primary NN Server, Run the below as root: sudo su hdfs -l -c 'hdfs dfsadmin -safemode enter' sudo su hdfs -l -c 'hdfs dfsadmin -saveNamespace' 5. Now Start the HDFS NN 6. Then the rest of the services of HDFS 7. Then YARN 8. Now rest of the services In my other environments they come up on their own. Could you assit me to fix this? FYI: We have active and Standby namenode for our environment. ... View more
Labels:

getting http response 503 for hive jobs running v...

by Explorer nehanirmal in Support Questions
‎11-03-2018 08:01 AM
‎11-03-2018 08:01 AM
Error: Caused by: org.apache.thrift.transport.TTransportException: HTTP Response Code: 503 Before this was getting error 500. Restarted hive and knox. Now getting 503 error message. Let me know if anyone knows the resolution for this. ... View more

Failed to start service via Cloudera manager, Erro...

by Explorer nehanirmal in Support Questions
‎07-27-2018 05:16 AM
‎07-27-2018 05:16 AM
Hi Team,   I am unable to start the services in the hadoop cluster. It displays error message as below: Command timed-out after 150 seconds  Cloudera-scm-agent cloudera-scm-server, cloudera-scm-db all are running on the server. In the CM the services aren't starting although.   I am guessing issuemight lie in Cloudera Agent log as Cloudera Manager console is accessible. Please look into the below mentioned logs and assist:   [27/Jul/2018 07:36:25 +0000] 2705 MainThread agent ERROR Failed to connect to previous supervisor. Traceback (most recent call last): File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.15.0-py2.7.egg/cmf/agent.py", line 2137, in find_or_start_supervisor self.get_supervisor_process_info() File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.15.0-py2.7.egg/cmf/agent.py", line 2281, in get_supervisor_process_info self.identifier = self.supervisor_client.supervisor.getIdentification() File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1587, in __request verbose=self.__verbose File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/supervisor-3.0-py2.7.egg/supervisor/xmlrpc.py", line 460, in request self.connection.request('POST', handler, request_body, self.headers) File "/usr/lib64/python2.7/httplib.py", line 1017, in request self._send_request(method, url, body, headers) File "/usr/lib64/python2.7/httplib.py", line 1051, in _send_request self.endheaders(body) File "/usr/lib64/python2.7/httplib.py", line 1013, in endheaders self._send_output(message_body) File "/usr/lib64/python2.7/httplib.py", line 864, in _send_output self.send(msg) File "/usr/lib64/python2.7/httplib.py", line 826, in send self.connect() File "/usr/lib64/python2.7/httplib.py", line 807, in connect self.timeout, self.source_address) File "/usr/lib64/python2.7/socket.py", line 571, in create_connection raise err error: [Errno 111] Connection refused [27/Jul/2018 07:36:25 +0000] 2705 MainThread tmpfs INFO Reusing mounted tmpfs at /run/cloudera-scm-agent/process [27/Jul/2018 07:36:25 +0000] 2705 Dummy-1 daemonize WARNING Stopping daemon. [27/Jul/2018 07:36:25 +0000] 2705 Dummy-1 agent INFO Stopping agent... [27/Jul/2018 07:36:25 +0000] 2705 Dummy-1 agent INFO No extant cgroups; unmounting any cgroup roots ... View more

Is HDP2.6.4 compatible with RHEL 7.5?

by Explorer nehanirmal in Support Questions
‎04-26-2018 01:36 PM
‎04-26-2018 01:36 PM
Hi All, Please let me know if we can install HDP 2.6.4 on RHEL 7.5? ... View more

Re: Impleted TLS, Unable to access CM after that

by Explorer nehanirmal in Support Questions
‎04-17-2018 02:52 AM
‎04-17-2018 02:52 AM
Rebooted the server. Was able to access the postgres after that. Deleted the web_tls after it ... View more

Impleted TLS, Unable to access CM after that

by Explorer nehanirmal in Support Questions
‎04-16-2018 10:15 AM
‎04-16-2018 10:15 AM
Made TLS implementation for Cloudera Manager. Unable to access CM after that.   Referred this link: https://community.cloudera.com/t5/Cloudera-Manager-Installation/how-to-rollback-cloudera-manager-tls-configuration-without-UI/td-p/46484 to revert the changes. Not able to access the embedded Postgres console to do so.   Getting the error: [root@hostname pgsql]# psql -U cloudera-scm -p 7432 -h localhost -d postgres psql: FATAL: the database system is shutting down   # lsof -i | grep LISTEN | grep cloudera-scm postgres 11232 cloudera-scm 3u IPv4 105319 0t0 TCP *:7432 (LISTEN) postgres 11232 cloudera-scm 4u IPv6 105320 0t0 TCP *:7432 (LISTEN) java 24548 cloudera-scm 243u IPv4 3988695 0t0 TCP hostname.com:us-srv (LISTEN) java 24548 cloudera-scm 244u IPv4 3988697 0t0 TCP *:rrac (LISTEN) java 24550 cloudera-scm 347u IPv4 3988161 0t0 TCP hostname.com:palace-6 (LISTEN) java 24550 cloudera-scm 348u IPv4 3989833 0t0 TCP hostname.com:d-s-n (LISTEN) java 24550 cloudera-scm 349u IPv4 3989832 0t0 TCP *:palace-5 (LISTEN) java 24587 cloudera-scm 237u IPv4 3987633 0t0 TCP *:7184 (LISTEN) java 24587 cloudera-scm 241u IPv4 3987635 0t0 TCP *:8084 (LISTEN) java 24587 cloudera-scm 245u IPv4 3987637 0t0 TCP *:7185 (LISTEN) java 24627 cloudera-scm 268u IPv4 3989842 0t0 TCP *:distinct32 (LISTEN) java 24627 cloudera-scm 272u IPv4 3989844 0t0 TCP hostname.com:distinct (LISTEN) java 24627 cloudera-scm 273u IPv4 3988199 0t0 TCP hostname.com:simplifymedia (LISTEN) java 24656 cloudera-scm 302u IPv4 3984366 0t0 TCP hostname.com:palace-4 (LISTEN) java 24656 cloudera-scm 308u IPv4 3986063 0t0 TCP *:palace-3 (LISTEN) java 24656 cloudera-scm 309u IPv4 3986068 0t0 TCP hostname.com:jamlink (LISTEN) java 24704 cloudera-scm 232u IPv4 3987626 0t0 TCP *:ezmeeting-2 (LISTEN)   Please suggest.     ... View more

Re: Unable to install Cloudera Manager on SUSE Lin...

by Explorer nehanirmal in Support Questions
‎03-07-2018 05:36 AM
‎03-07-2018 05:36 AM
Suddenly my cloudera-scm-server has gone dead, when i restart it it again goes to being dead in 3 secs.   Getting the below  cloudera-scm-server.log output:   JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera_backup Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.cloudera.server.cmf.TrialState': Cannot resolve reference to bean 'entityManagerFactoryBean' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106) at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:616) at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:148) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1003) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:907) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:293) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:290) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425) at com.cloudera.server.cmf.Main.bootstrapSpringContext(Main.java:393) at com.cloudera.server.cmf.Main.<init>(Main.java:243) at com.cloudera.server.cmf.Main.main(Main.java:216) Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactoryBean': FactoryBean threw exception on object creation; nested exception is javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:149) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:102) at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1440) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:247) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:192) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322) ... 17 more Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387) at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310) at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1397) at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:62) at com.cloudera.enterprise.AbstractWrappedEntityManager.beginForRollbackAndReadonly(AbstractWrappedEntityManager.java:89) at com.cloudera.enterprise.dbutil.DbUtil.isInnoDbEnabled(DbUtil.java:549) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.checkMysqlTableEngineType(EntityManagerFactoryBean.java:139) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:122) at com.cloudera.server.cmf.bootstrap.EntityManagerFactoryBean.getObject(EntityManagerFactoryBean.java:65) at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:142) ... 22 more Caused by: org.hibernate.exception.GenericJDBCException: Could not open connection at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125) at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110) at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:221) at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.getConnection(LogicalConnectionImpl.java:157) at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.doBegin(JdbcTransaction.java:67) at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.begin(AbstractTransactionImpl.java:160) at org.hibernate.internal.SessionImpl.beginTransaction(SessionImpl.java:1426) at org.hibernate.ejb.TransactionImpl.begin(TransactionImpl.java:59) ... 28 more Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database! at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106) at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:529) at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128) at org.hibernate.service.jdbc.connections.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:84) at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:292) at org.hibernate.engine.jdbc.internal.LogicalConnectionImpl.obtainConnection(LogicalConnectionImpl.java:214) ... 33 more Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source. at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1319) at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557) at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477) at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525) ... 37 more ... View more

Re: sqlContext show it cannot connect Derby databa...

by Explorer nehanirmal in Support Questions
‎02-20-2018 12:50 AM
‎02-20-2018 12:50 AM
Hive Gateway is present on the host I am trying to run the spark job. ... View more

Re: Getting error in Spark-Shell while using hive ...

by Explorer nehanirmal in Support Questions
‎02-20-2018 12:47 AM
‎02-20-2018 12:47 AM
hiveContext.sql("select * from test.automation_inp3").show 18/02/19 08:53:55 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 18/02/19 08:53:55 INFO ObjectStore: ObjectStore, initialize called 18/02/19 08:53:55 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-api-jdo-3.2.1.jar." 18/02/19 08:53:55 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-core-3.2.2.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-core-3.2.10.jar." 18/02/19 08:53:55 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-rdbms-3.2.1.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/cloudera/parcels/CDH-5.4.10-1.cdh5.4.10.p0.16/jars/datanucleus-rdbms-3.2.9.jar." 18/02/19 08:53:55 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 18/02/19 08:53:55 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 18/02/19 08:53:56 WARN HiveMetaStore: Retrying creating default database after error: Error creating transactional connection factory javax.jdo.JDOFatalInternalException: Error creating transactional connection factory at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:587) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) at java.security.AccessController.doPrivileged(Native Method) at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166) at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:390) at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:419) at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:314) at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:281) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:56) at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:65) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:584) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:562) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:611) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:453) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5716) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:198) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1486) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:64) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:74) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2895) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2914) at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3139) at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:205) at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:192) at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:302) at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:263) at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:238) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:491) at org.apache.spark.sql.hive.HiveContext.sessionState$lzycompute(HiveContext.scala:229) at org.apache.spark.sql.hive.HiveContext.sessionState(HiveContext.scala:225) at org.apache.spark.sql.hive.HiveContext.hiveconf$lzycompute(HiveContext.scala:241) at org.apache.spark.sql.hive.HiveContext.hiveconf(HiveContext.scala:240) at org.apache.spark.sql.hive.HiveContext.sql(HiveContext.scala:86) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:29) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:34) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:36) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:38) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:40) at $line25.$read$$iwC$$iwC$$iwC$$iwC$$iwC.<init>(<console>:42) at $line25.$read$$iwC$$iwC$$iwC$$iwC.<init>(<console>:44) at $line25.$read$$iwC$$iwC$$iwC.<init>(<console>:46) at $line25.$read$$iwC$$iwC.<init>(<console>:48) at $line25.$read$$iwC.<init>(<console>:50) at $line25.$read.<init>(<console>:52) at $line25.$read$.<init>(<console>:56) at $line25.$read$.<clinit>(<console>) at $line25.$eval$.<init>(<console>:7) at $line25.$eval$.<clinit>(<console>) at $line25.$eval.$print(<console>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338) at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:856) at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:901) at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:813) at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:656) at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:664) at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$loop(SparkILoop.scala:669) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:996) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:944) at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply(SparkILoop.scala:944) at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop.scala:944) at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1058) at org.apache.spark.repl.Main$.main(Main.scala:31) at org.apache.spark.repl.Main.main(Main.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:569) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:166) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:189) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:110) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) NestedThrowablesStackTrace: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631) at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:325) at org.datanucleus.store.AbstractStoreManager.registerConnectionFactory(AbstractStoreManager.java:282) at org.datanucleus.store.AbstractStoreManager.<init>(AbstractStoreManager.java:240) at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:286) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631) at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301) at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187) at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333) at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) at java.security.AccessController.doPrivileged(Native Method) at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166) at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:390) at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:419) at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:314) at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:281) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:56) at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:65) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:584) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:562) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:611) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:453) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:78) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:84) at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5716) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:198) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ... View more

Getting error in Spark-Shell while using hive cont...

by Explorer nehanirmal in Support Questions
‎02-19-2018 10:28 PM
‎02-19-2018 10:28 PM
Getting error: HiveMetaStore: Retrying creating default database after error: Error creating transactional connection factory javax.jdo.JDOFatalInternalException: Error creating transactional connection factory   Query run: import org.apache.spark.sql.hive.HiveContext val hiveContext = new HiveContext(sc) import hiveContext.implicits._ import hiveContext.sql   hiveContext.sql("select * from test.test_3").show ... View more

Re: Unable to install Cloudera Manager on SUSE Lin...

by Explorer nehanirmal in Support Questions
‎02-08-2018 03:44 AM
‎02-08-2018 03:44 AM
Thanks, Chose the last option and went ahead, installation happened successfully. Not sure of any drawbacks it will lead to. ... View more

Re: Unable to install Cloudera Manager on SUSE Lin...

by Explorer nehanirmal in Support Questions
‎02-07-2018 08:37 AM
‎02-07-2018 08:37 AM
Manually installed mysql and tried to do path B install, getting the below error:   BEGIN zypper --gpg-auto-import-keys -n se -i --match-exact -t package cloudera-manager-agent | grep -E '^i[[:space:]]*\|[[:space:]]*cloudera-manager-agent' END (1) BEGIN zypper --gpg-auto-import-keys -n se --match-exact -t package cloudera-manager-agent Loading repository data... Reading installed packages... S | Name | Summary | Type --+------------------------+----------------------------+-------- | cloudera-manager-agent | The Cloudera Manager Agent | package END (0) BEGIN zypper info cloudera-manager-agent | grep -E 'Version[[:space:]]*:[[:space:]]5.14.0-.*\.25' Version: 5.14.0-1.cm5140.p0.25.sles11 END (0) BEGIN zypper --gpg-auto-import-keys -n in cloudera-manager-agent Loading repository data... Reading installed packages... Resolving package dependencies... Problem: nothing provides apache2 needed by cloudera-manager-agent-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 1: do not install cloudera-manager-agent-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 2: break cloudera-manager-agent-5.14.0-1.cm5140.p0.25.sles11.x86_64 by ignoring some of its dependencies Choose from above solutions by number or cancel [1/2/c] (c): c END (4) remote package cloudera-manager-agent could not be installed, giving up waiting for rollback request   Please suggest any fix. ... View more

Re: Unable to install Cloudera Manager on SUSE Lin...

by Explorer nehanirmal in Support Questions
‎02-05-2018 09:01 AM
‎02-05-2018 09:01 AM
Did the same as you mentioned, but still the same error is popping up: hostname:~ # zypper addrepo http://download.opensuse.org/repositories/server:database:postgresql/SLE_11_SP4/server:database:postgresql.repo Adding repository 'PostgreSQL and related packages (SLE_11_SP4)' [done] Repository 'PostgreSQL and related packages (SLE_11_SP4)' successfully added Enabled: Yes Autorefresh: No GPG check: Yes URI: http://download.opensuse.org/repositories/server:/database:/postgresql/SLE_11_SP4/ zypper refresh hostname:~ # zypper refresh Retrieving repository 'PostgreSQL and related packages (SLE_11_SP4)' metadata [\] New repository or package signing key received: Key ID: 562111AC05905EA8 Key Name: server:database OBS Project <server:database@build.opensuse.org> Key Fingerprint: 116EB86331583E47E63CDF4D562111AC05905EA8 Key Created: Thu 02 Feb 2017 08:09:57 AM EST Key Expires: Sat 13 Apr 2019 09:09:57 AM EDT Repository: PostgreSQL and related packages (SLE_11_SP4) Do you want to reject the key, trust temporarily, or trust always? [r/t/a/? shows all options] (r): a Retrieving repository 'PostgreSQL and related packages (SLE_11_SP4)' metadata [done] Building repository 'PostgreSQL and related packages (SLE_11_SP4)' cache [done] All repositories have been refreshed. hostname:~ # ls -l /etc/zypp/repos.d/ | grep postgres -rw-r--r-- 1 root root 330 Feb 5 11:51 server_database_postgresql.repo hostname:~ # zypper clean --all All repositories have been cleaned up.   Still getting the same error:   hostname:/var/log/cloudera-manager-installer # ll total 28 -rw-r--r-- 1 root root 56 Feb 5 11:53 0.check-selinux.log -rw-r--r-- 1 root root 0 Feb 5 11:54 1.install-repo-pkg.log -rw-r--r-- 1 root root 784 Feb 5 11:54 2.install-oracle-j2sdk1.7.log -rw-r--r-- 1 root root 1368 Feb 5 11:56 3.install-cloudera-manager-server.log -rw-r--r-- 1 root root 33 Feb 5 11:56 4.check-for-systemd.log -rw-r--r-- 1 root root 496 Feb 5 11:56 5.install-cloudera-manager-server-db-2.log -rw-r--r-- 1 root root 51 Feb 5 11:56 6.remove-cloudera-manager-server.log -rw-r--r-- 1 root root 0 Feb 5 11:56 7.remove-cloudera-manager-daemons.log -rw-r--r-- 1 root root 106 Feb 5 11:56 8.remove-cloudera-manager-repository.log hostname:/var/log/cloudera-manager-installer # pwd /var/log/cloudera-manager-installer hostname:/var/log/cloudera-manager-installer # cat 5.install-cloudera-manager-server-db-2.log Loading repository data... Reading installed packages... Resolving package dependencies... Problem: nothing provides postgresql-server >= 8.4 needed by cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 1: do not install cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 2: break cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 by ignoring some of its dependencies Choose from above solutions by number or cancel [1/2/c] (c): c ... View more

Unable to install Cloudera Manager on SUSE Linux E...

by Explorer nehanirmal in Support Questions
‎02-05-2018 07:19 AM
‎02-05-2018 07:19 AM
Unable to install Cloudera Manager on SUSE Linux Enterprise Server 11 (x86_64)   getting the below error in the /var/log/cloudera-manager-installer/5.install-cloudera-manager-server-db-2.log:  Problem: nothing provides postgresql-server >= 8.4 needed by cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 1: do not install cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 Solution 2: break cloudera-manager-server-db-2-5.14.0-1.cm5140.p0.25.sles11.x86_64 by ignoring some of its dependencies. ... View more

Re: Running shell scripts in oozie using hue

by Explorer nehanirmal in Support Questions
‎01-29-2018 07:16 AM
‎01-29-2018 07:16 AM
So,   1.  I should uncheck thie below? Limit Nonsecure Container Executor Users --> YARN (MR2 Included) (Service-Wide) { currently the check box is selected}   2. what should I put instead of nobody for the below? UNIX User for Nonsecure Mode with Linux Container Executor yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user YARN (MR2 Included) (Service-Wide)   ---> Nobody      3. Is there something more I should change?   4. Kerberos is not enabled in the system. ... View more

Running shell scripts in oozie using hue

by Explorer nehanirmal in Support Questions
‎01-29-2018 06:43 AM
‎01-29-2018 06:43 AM
Running the below script: #!/bin/bash ssh user@host.abcd.com "if [ -f /data/home/user/datafiles/Test.csv ] ; then echo 1 ; else echo 0 ; fi;"   Set up passwordless ssh for 'user' too.    Getting the below error: Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password). Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]   Please Assist! ... View more
Labels:

Not able to view hive databases in impala-shell or...

by Explorer nehanirmal in Support Questions
‎01-16-2018 03:34 AM
‎01-16-2018 03:34 AM
Hi,   Not able to view hive databases in impala-shell or Hue Impala Editor.   Please help me fix this.   Sentry has been enabled but kerberos is not set for this environment. ... View more

Cannot create database or perform any hive operati...

by Explorer nehanirmal in Support Questions
‎01-11-2018 09:22 AM
‎01-11-2018 09:22 AM
Logged to beeline: beeline !connect jdbc:hive2://<server name>:10000/default username: hive password: hive   Able to perform grant and create role but not able to do anything else. Hive should have super user privileges Error: Error: Error while compiling statement: FAILED: SemanticException No valid privileges User hive does not have privileges for CREATEDATABASE The required privileges: Server=server1->action=*; (state=42000,code=40000)   Kindly Assist. ... View more
Labels:

Re: HTTPS configuration for ambari server

by Explorer nehanirmal in Support Questions
‎12-01-2017 04:55 AM
‎12-01-2017 04:55 AM
@Jay Kumar SenSharma It throws an exception, need to add it to the exception list to access it. When enable https options comes up, we just need to upload the server cert out of the three certs in the keystore, right? ... View more

HTTPS configuration for ambari server

by Explorer nehanirmal in Support Questions
‎11-30-2017 11:20 AM
‎11-30-2017 11:20 AM
Hi @Jay Kumar SenSharma In regards to your link: https://community.hortonworks.com/articles/39865/enabling-https-for-ambariserver-and-troubleshootin.html I have configured https for my ambari -server, but when opening ambari Ui in the web browser, it shows connection not secure. Can you help me to fix this issue? I have 3 certificates- server certificate, intermediate certificate and root certificate. ... View more
Labels:

Re: Disabling TLSv1 & TLS1.1 - Enabling TLSv1.2

by Explorer nehanirmal in Support Questions
‎11-20-2017 10:29 AM
‎11-20-2017 10:29 AM
@Vipin Rathor Yes, Added this property on one of the server that had agent file. Still it was also not able to communicate with the ambari-server. ... View more

Re: Disabling TLSv1 & TLS1.1 - Enabling TLSv1.2

by Explorer nehanirmal in Support Questions
‎11-20-2017 05:09 AM
‎11-20-2017 05:09 AM
@Vipin Rathor I am facing the same issue of agents losing heartbeat, even though the version of ambari being used here is 2.5.0. Please suggest. ... View more

Re: Ranger Hive Policy Question

by Explorer nehanirmal in Support Questions
‎11-13-2017 04:45 AM
‎11-13-2017 04:45 AM
Hi, I am using zeppelin to run hive queries. (Ranger is enabled) With or without user impersonation, when I try running the queries I get the below error: Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [anonymous] does not have [USE] privilege on [null] This works fine when in default user I put some username. This also works fine for the user if using Hive CLI. Can anyone please help? ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-10-2017 04:35 AM
‎11-10-2017 04:35 AM
@Aditya Sirna did you find anything on this ? ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-08-2017 04:11 AM
‎11-08-2017 04:11 AM
Details of siro_ini_content [urls] # This section is used for url-based security. # You can secure interpreter, configuration and credential information by urls. Comment or uncomment the below urls that you want to hide. # anon means the access is anonymous. # authc means Form based Auth Security # To enfore security, comment the line below and uncomment the next one #/api/version = anon #/api/interpreter/** = authc, roles[admin] #/api/configurations/** = authc, roles[admin] #/api/credential/** = authc, roles[admin] #/** = anon #/** = authc /api/configurations/** = authc, roles[admin] /api/credential/** = authc, roles[admin] /** = authc ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-08-2017 04:09 AM
‎11-08-2017 04:09 AM
@Aditya Sirna , @Geoffrey Shelton Okot Should this line be commented or not? #/api/version = anon What is this exactly doing? The problem is that the requests are getting submitted as anonymous irrespective to keeping the user impersonation option checked or unchecked. It's an Access Control Exception. Please suggest. ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-07-2017 07:35 AM
‎11-07-2017 07:35 AM
@Aditya Sirna, This property is set true for both in security part and in hive interactive as well. In hive-interactive this is set as default setting. Although using hiveserver2. ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-07-2017 06:51 AM
‎11-07-2017 06:51 AM
For hive.server2.enable.doAs, hive-interactive site has "Run as end user instead of Hive user=true" When running the queries from beeline for the same user it works fine so permissions are correct. Jdbc interpreter does not have any of the above parameter. Although tried with these as well. It keeps taking user as anonymous if no default user is specified. And anonymous has no access. In Zeppelin config: "zeppelin.anonymous.allowed=false" is also set. ... View more

Re: Zeppelin Interpreter User Impersonation proper...

by Explorer nehanirmal in Support Questions
‎11-07-2017 04:04 AM
‎11-07-2017 04:04 AM
@Jay Kumar SenSharma if you could help me on this? ... View more

Zeppelin Interpreter User Impersonation property n...

by Explorer nehanirmal in Support Questions
‎11-07-2017 04:00 AM
‎11-07-2017 04:00 AM
Created %hive interpreter with user impersonation enabled and submitting queries getting this error: user [anonymous] does not have [USE] privilege on [null] Created %jdbc(hive) and submitting jobs, the user is able to access everything as the query is running with hive as a user(Ranger permissions set for hive to access everything) Please help me configure this so that the user logged in is able to access only those tables which it has access for in Ranger policies. ... View more
Contact Me
Online
Offline
Last Visited
‎08-23-2018 10:39 AM
Public Statistics
Date Registered ‎04-17-2017 03:39 AM
Last Visited ‎08-23-2018 10:39 AM
Total Messages Posted 40