@silvan    I did a fresh install of Tez followed by HMS followed by Hive on Tez.   But Hive on TEZ not coming up and throws below error, Kindly suggest?   - Is there any configuration property needs to updated in Tez or HMS or at some place?   Error:   org.apache.hive.service.ServiceException: org.apache.hive.service.ServiceException: Unable to setup tez session pool at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:690) ~[hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1016) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:137) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1294) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1138) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.7.0.3.0-79.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.7.0.3.0-79.jar:?] Caused by: org.apache.hive.service.ServiceException: Unable to setup tez session pool       2020-04-16 15:24:13,671 ERROR org.apache.hive.service.server.HiveServer2: [main]: Error starting Tez sessions: org.apache.hive.service.ServiceException: Unable to setup tez session pool at org.apache.hive.service.server.HiveServer2.initAndStartTezSessionPoolManager(HiveServer2.java:793) ~[hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.startOrReconnectTezSessions(HiveServer2.java:763) ~[hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:687) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1016) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.access$1400(HiveServer2.java:137) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1294) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1138) [hive-service-3.1.2000.7.0.3.0-79.jar:3.1.2000.7.0.3.0-79] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] at org.apache.hadoop.util.RunJar.run(RunJar.java:318) [hadoop-common-3.1.1.7.0.3.0-79.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [hadoop-common-3.1.1.7.0.3.0-79.jar:?] Caused by: org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1587049942856_0006 failed 2 times due to AM Container for appattempt_1587049942856_0006_000002 exited with exitCode: 1   - VM     ... View more

@bgooley    Cloudera Navigator Metastore needs TLS password to be set in CM UI otherwise it wont start.   Thanks for the help to fix the issue.   - Vijay M ... View more

@bgooley    Any suggestion?   - Vijay M ... View more

@bgooley    I tried to restart and there is no process directory getscreated for Navigator Metastore.   I had tail the cloudera agent log for server where Navigator Metadata is running. Cloudera agent throws below logs when i restarted Navigator Metastore.   Cloudera Agent log:   [25/Jun/2019 08:30:32 +0000] 2813189 CP Server Thread-5 _cplogging INFO 10.26.132.165 - - [25/Jun/2019:08:30:32] "GET /heartbeat HTTP/1.1" 200 2 "" "NING/1.0" [25/Jun/2019 08:30:34 +0000] 2813189 MainThread navigator_plugin INFO Scheduling a refresh for Metadata Plugin for conf.cloudera.spark2_on_yarn with count 0 pipelines names []. [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin __init__ INFO Read metadata config for role GATEWAY. Metadata server url: https://a301-7781-0339.ldn.swissbank.com:7187 [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin navigator_plugin INFO Piplines updated for Metadata Plugin: [] [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin navigator_plugin INFO Refreshing Metadata Plugin for conf.cloudera.spark2_on_yarn with count 0 pipelines names []. [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin navigator_thread INFO Log entry poll interval: 5 [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin navigator_thread INFO Navigator server url: https://a301-7781-0339.ldn.swissbank.com:7187 [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin inotify_event_processor INFO Setting watch for directories: /var/log/hive/lineage [25/Jun/2019 08:30:37 +0000] 2813189 Metadata-Plugin navigator_logs INFO 9 navigator log files with prefix hive-HIVESERVER2-220e59e0745b944070fe360a2cc717c7 for role hive-HIVESERVER2-220e59e0745b944070fe360a2cc717c7 exist in navigator log directory /var/log/hive/lineage     Cloudera Sever throws below logs.   2019-06-25 08:33:07,341 INFO 439059796@scm-web-6908:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Start this Navigator Metadata Server on role Navigator Metadata Server (a301-8744-7259) 2019-06-25 08:33:07,372 INFO 439059796@scm-web-6908:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing role command Start BasicCmdArgs{args=[]}. Service: DbService{id=2, name=mgmt} Role: DbRole{id=324, name=mgmt-NAVIGATORMETASERVER-220e59e0745b944070fe360a2cc717c7, hostName=a301-8744-7259.ldn.swissbank.com} 2019-06-25 08:33:07,372 INFO 439059796@scm-web-6908:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence 2019-06-25 08:33:07,372 INFO 439059796@scm-web-6908:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Start a role 2019-06-25 08:33:07,884 WARN 439059796@scm-web-6908:com.cloudera.cmf.service.GenericBringUpRoleCommand: Unexpected exception in command execution java.util.NoSuchElementException at com.google.common.collect.Iterators$1.next(Iterators.java:77) at com.google.common.collect.Iterators.getOnlyElement(Iterators.java:325) at com.google.common.collect.Iterables.getOnlyElement(Iterables.java:271) at com.cloudera.cmf.service.mgmt.RoleSSLUtils.getTrustStoreServerOpts(RoleSSLUtils.java:43) at com.cloudera.cmf.service.mgmt.NavMetaServerRoleHandler.getEnvironmentForConcreteRole(NavMetaServerRoleHandler.java:99) at com.cloudera.cmf.service.mgmt.BaseMgmtRoleHandler.getEnvironmentForRole(BaseMgmtRoleHandler.java:123) at com.cloudera.cmf.service.AbstractDaemonRoleHandler.getEnvironment(AbstractDaemonRoleHandler.java:347) at com.cloudera.cmf.service.mgmt.BaseMgmtRoleHandler.makeProcess(BaseMgmtRoleHandler.java:73) at com.cloudera.cmf.service.GenericBringUpRoleCommand$RoleBringUpCmdWork.doWork(GenericBringUpRoleCommand.java:203) at com.cloudera.cmf.command.flow.CmdStep.doWork(CmdStep.java:177) at com.cloudera.cmf.command.flow.SeqCmdWork.doWork(SeqCmdWork.java:107) at com.cloudera.cmf.command.flow.CmdStep.doWork(CmdStep.java:177) at com.cloudera.cmf.command.flow.SeqFlowCmd.run(SeqFlowCmd.java:117) at com.cloudera.cmf.command.CmdWorkCommand.execute(CmdWorkCommand.java:94) at com.cloudera.cmf.service.ServiceHandlerRegistry.executeRoleCommandHelper(ServiceHandlerRegistry.java:943) at com.cloudera.cmf.service.ServiceHandlerRegistry.executeRoleCommand(ServiceHandlerRegistry.java:898) at com.cloudera.cmf.command.flow.CmdWorkCtx.execRoleCmd(CmdWorkCtx.java:145) at com.cloudera.cmf.command.flow.work.ExecRoleCmdWork.doWork(ExecRoleCmdWork.java:58) at com.cloudera.cmf.command.flow.CmdStep.doWork(CmdStep.java:177) at com.cloudera.cmf.command.flow.SeqCmdWork.doWork(SeqCmdWork.java:107) at com.cloudera.cmf.command.flow.CmdStep.doWork(CmdStep.java:177) at com.cloudera.cmf.command.flow.SeqFlowCmd.run(SeqFlowCmd.java:117) at com.cloudera.cmf.service.AbstractRestartCommands$GenericRestartRoleCommand.executeImpl(AbstractRestartCommands.java:187) at com.cloudera.cmf.service.AbstractRoleCommand.execute(AbstractRoleCommand.java:64) at com.cloudera.cmf.service.AbstractRoleCommand.execute(AbstractRoleCommand.java:24) at com.cloudera.cmf.service.ServiceHandlerRegistry.executeRoleCommandHelper(ServiceHandlerRegistry.java:943) at com.cloudera.cmf.service.ServiceHandlerRegistry.executeRoleCommand(ServiceHandlerRegistry.java:898) at com.cloudera.cmf.service.ServiceHandlerRegistry.executeRoleCommand(ServiceHandlerRegistry.java:893) at com.cloudera.server.cmf.components.OperationsManagerImpl.executeRoleCommand(OperationsManagerImpl.java:1811) at com.cloudera.server.web.cmf.InstancesController.internalRoleInstancesDispatch(InstancesController.java:469) at com.cloudera.server.web.cmf.InstancesController.access$100(InstancesController.java:76) at com.cloudera.server.web.cmf.InstancesController$3.apply(InstancesController.java:437) at com.cloudera.server.web.cmf.InstancesController$3.apply(InstancesController.java:434) at com.cloudera.server.web.cmf.InvocationUtil$1.call(InvocationUtil.java:36) at com.cloudera.server.common.RetryWrapper.executeWithRetry(RetryWrapper.java:32) at com.cloudera.server.common.RetryUtils.executeWithRetryHelper(RetryUtils.java:210) at com.cloudera.server.common.RetryUtils.executeWithRetry(RetryUtils.java:108) at com.cloudera.server.web.cmf.InvocationUtil.invoke(InvocationUtil.java:32) at com.cloudera.server.web.cmf.InstancesController.roleInstancesDispatch(InstancesController.java:433) at com.cloudera.server.web.cmf.InstancesController$$FastClassByCGLIB$$17ed6885.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191) at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:61) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621) at com.cloudera.server.web.cmf.InstancesController$$EnhancerByCGLIB$$16b0d4df.roleInstancesDispatch(<generated>) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:176) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:436) at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:424) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:669) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:585) at javax.servlet.http.HttpServlet.service(HttpServlet.java:595) at javax.servlet.http.HttpServlet.service(HttpServlet.java:668) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) at org.mortbay.servlet.UserAgentFilter.doFilter(UserAgentFilter.java:78) at org.mortbay.servlet.GzipFilter.doFilter(GzipFilter.java:131) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at com.jamonapi.http.JAMonServletFilter.doFilter(JAMonServletFilter.java:48) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at com.cloudera.enterprise.JavaMelodyFacade$MonitoringFilter.doFilter(JavaMelodyFacade.java:109) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:146) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:182) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:125) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:767) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.handler.StatisticsHandler.handle(StatisticsHandler.java:53) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2019-06-25 08:33:07,885 ERROR 439059796@scm-web-6908:com.cloudera.cmf.command.flow.WorkOutputs: CMD id: 7269 Role failed to start due to error null. 2019-06-25 08:33:08,032 WARN 439059796@scm-web-6908:com.cloudera.cmf.service.GenericBringUpRoleCommand: Role 324/mgmt-NAVIGATORMETASERVER-220e59e0745b944070fe360a2cc717c7 Process cloudera-mgmt-NAVIGATORMETASERVER does not exist - unable to update status 2019-06-25 08:33:08,032 INFO 439059796@scm-web-6908:com.cloudera.cmf.model.DbCommand: Command 7269(Start) has completed. finalstate:FINISHED, success:false, msg:Failed to start role. 2019-06-25 08:33:08,032 ERROR 439059796@scm-web-6908:com.cloudera.cmf.command.flow.WorkOutputs: CMD id: 7269 Failed to start role. 2019-06-25 08:33:08,032 INFO 439059796@scm-web-6908:com.cloudera.cmf.model.DbCommand: Command 7268(Restart) has completed. finalstate:FINISHED, success:false, msg:Failed to start role. 2019-06-25 08:33:08,447 INFO 439059796@scm-web-6908:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command ProcessStalenessCheckCommand BasicCmdArgs{args=[First reason why: com.cloudera.cmf.model.DbConfig.valueForDb (#814) has changed]}. 2019-06-25 08:33:08,448 INFO 439059796@scm-web-6908:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence 2019-06-25 08:33:08,448 INFO 439059796@scm-web-6908:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Configuration Staleness Check 2019-06-25 08:33:09,456 INFO avro-servlet-hb-processor-2:com.cloudera.server.common.AgentAvroServlet: (23 skipped) AgentAvroServlet: heartbeat processing stats: average=1136ms, min=719ms, max=2947ms. 2019-06-25 08:33:10,359 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Running staleness check with FULL_CHECK for 44/74 roles. 2019-06-25 08:33:10,432 INFO ProcessStalenessDetector-0:com.cloudera.cmf.service.config.components.ProcessStalenessDetector: Total staleness check tasks: 77   - Vijay M ... View more

@lwang    Below 2 Directories have generated.   drwxr-x--x. 3 root root 180 Jun 21 12:34 572-MGMT.NAVIGATOR-test-db-connection drwxr-x--x. 3 root root 180 Jun 21 12:34 571-MGMT.NAVIGATORMETASERVER-test-db-connection   I checked in both directories for logs, Below snap for it. But its not helping much.   Directory: 571-MGMT.NAVIGATORMETASERVER-test-db-connection   root@abc% pwd /run/cloudera-scm-agent/process/571-MGMT.NAVIGATORMETASERVER-test-db-connection/logs root@abc% ls -lrt total 8 -rw-r--r--. 1 root root 3676 Jun 20 12:25 stderr.log -rw-r--r--. 1 root root 394 Jun 20 12:25 stdout.log   root@abc% cat stderr.log Thu Jun 20 12:25:58 BST 2019 + date + date Thu Jun 20 12:25:58 BST 2019 ++ dirname /usr/lib64/cmf/service/dbconnection/test_db_connection.sh + cloudera_config=/usr/lib64/cmf/service/dbconnection ++ cd /usr/lib64/cmf/service/dbconnection/../common ++ pwd + cloudera_config=/usr/lib64/cmf/service/common + . /usr/lib64/cmf/service/common/cloudera-config.sh ++ export JAVA_HOME=/app/bds/java ++ JAVA_HOME=/app/bds/java ++ export 'JAVA_TOOL_OPTIONS=-Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf' ++ JAVA_TOOL_OPTIONS='-Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf' ++ export KRB5_CONFIG=/etc/krb5_bds.conf ++ KRB5_CONFIG=/etc/krb5_bds.conf ++ set -x + locate_java_home + locate_java_home_no_verify + JAVA6_HOME_CANDIDATES=('/usr/lib/j2sdk1.6-sun' '/usr/lib/jvm/java-6-sun' '/usr/lib/jvm/java-1.6.0-sun-1.6.0' '/usr/lib/jvm/j2sdk1.6-oracle' '/usr/lib/jvm/j2sdk1.6-oracle/jre' '/usr/java/jdk1.6' '/usr/java/jre1.6') + local JAVA6_HOME_CANDIDATES + OPENJAVA6_HOME_CANDIDATES=('/usr/lib/jvm/java-1.6.0-openjdk' '/usr/lib/jvm/jre-1.6.0-openjdk') + local OPENJAVA6_HOME_CANDIDATES + JAVA7_HOME_CANDIDATES=('/usr/java/jdk1.7' '/usr/java/jre1.7' '/usr/lib/jvm/j2sdk1.7-oracle' '/usr/lib/jvm/j2sdk1.7-oracle/jre' '/usr/lib/jvm/java-7-oracle') + local JAVA7_HOME_CANDIDATES + OPENJAVA7_HOME_CANDIDATES=('/usr/lib/jvm/java-1.7.0-openjdk' '/usr/lib/jvm/java-7-openjdk') + local OPENJAVA7_HOME_CANDIDATES + JAVA8_HOME_CANDIDATES=('/usr/java/jdk1.8' '/usr/java/jre1.8' '/usr/lib/jvm/j2sdk1.8-oracle' '/usr/lib/jvm/j2sdk1.8-oracle/jre' '/usr/lib/jvm/java-8-oracle') + local JAVA8_HOME_CANDIDATES + OPENJAVA8_HOME_CANDIDATES=('/usr/lib/jvm/java-1.8.0-openjdk' '/usr/lib/jvm/java-8-openjdk' '/usr/lib64/jvm/java-1.8.0-openjdk' '/usr/lib64/jvm/java-8-openjdk') + local OPENJAVA8_HOME_CANDIDATES + MISCJAVA_HOME_CANDIDATES=('/Library/Java/Home' '/usr/java/default' '/usr/lib/jvm/default-java' '/usr/lib/jvm/java-openjdk' '/usr/lib/jvm/jre-openjdk') + local MISCJAVA_HOME_CANDIDATES + case ${BIGTOP_JAVA_MAJOR} in + JAVA_HOME_CANDIDATES=(${JAVA7_HOME_CANDIDATES[@]} ${JAVA8_HOME_CANDIDATES[@]} ${JAVA6_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA7_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]} ${OPENJAVA6_HOME_CANDIDATES[@]}) + '[' -z /app/bds/java ']' + verify_java_home + '[' -z /app/bds/java ']' + echo JAVA_HOME=/app/bds/java + source_parcel_environment + '[' '!' -z '' ']' + JAVA=/app/bds/java/bin/java + [[ -z /usr/share/cmf ]] + MGMT_CLASSPATH='/usr/share/cmf/lib/*' + JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar + MGMT_CLASSPATH='/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' ++ pwd + MGMT_CLASSPATH='/run/cloudera-scm-agent/process/571-MGMT.NAVIGATORMETASERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' + MGMT_JAVA_OPTS='-Djava.net.preferIPv4Stack=true ' + exec /app/bds/java/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/run/cloudera-scm-agent/process/571-MGMT.NAVIGATORMETASERVER-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties Picked up JAVA_TOOL_OPTIONS: -Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf   root@abc% cat stdout.log Thu Jun 20 12:25:58 BST 2019 Current working directory: /run/cloudera-scm-agent/process/571-MGMT.NAVIGATORMETASERVER-test-db-connection Launching one-off process: /usr/lib64/cmf/service/dbconnection/test_db_connection.sh db.properties Thu Jun 20 12:25:58 BST 2019 JAVA_HOME=/app/bds/java [ main] DbCommandExecutor INFO Successfully connected to database.         Directory: /run/cloudera-scm-agent/process/572-MGMT.NAVIGATOR-test-db-connection/logs   root@abc% pwd /run/cloudera-scm-agent/process/572-MGMT.NAVIGATOR-test-db-connection/logs root@abc% ls -lrt total 8 -rw-r--r--. 1 root root 3656 Jun 20 12:25 stderr.log -rw-r--r--. 1 root root 384 Jun 20 12:25 stdout.log     root@abc% cat stderr.log Thu Jun 20 12:25:58 BST 2019 + date + date Thu Jun 20 12:25:58 BST 2019 ++ dirname /usr/lib64/cmf/service/dbconnection/test_db_connection.sh + cloudera_config=/usr/lib64/cmf/service/dbconnection ++ cd /usr/lib64/cmf/service/dbconnection/../common ++ pwd + cloudera_config=/usr/lib64/cmf/service/common + . /usr/lib64/cmf/service/common/cloudera-config.sh ++ export JAVA_HOME=/app/bds/java ++ JAVA_HOME=/app/bds/java ++ export 'JAVA_TOOL_OPTIONS=-Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf' ++ JAVA_TOOL_OPTIONS='-Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf' ++ export KRB5_CONFIG=/etc/krb5_bds.conf ++ KRB5_CONFIG=/etc/krb5_bds.conf ++ set -x + locate_java_home + locate_java_home_no_verify + JAVA6_HOME_CANDIDATES=('/usr/lib/j2sdk1.6-sun' '/usr/lib/jvm/java-6-sun' '/usr/lib/jvm/java-1.6.0-sun-1.6.0' '/usr/lib/jvm/j2sdk1.6-oracle' '/usr/lib/jvm/j2sdk1.6-oracle/jre' '/usr/java/jdk1.6' '/usr/java/jre1.6') + local JAVA6_HOME_CANDIDATES + OPENJAVA6_HOME_CANDIDATES=('/usr/lib/jvm/java-1.6.0-openjdk' '/usr/lib/jvm/jre-1.6.0-openjdk') + local OPENJAVA6_HOME_CANDIDATES + JAVA7_HOME_CANDIDATES=('/usr/java/jdk1.7' '/usr/java/jre1.7' '/usr/lib/jvm/j2sdk1.7-oracle' '/usr/lib/jvm/j2sdk1.7-oracle/jre' '/usr/lib/jvm/java-7-oracle') + local JAVA7_HOME_CANDIDATES + OPENJAVA7_HOME_CANDIDATES=('/usr/lib/jvm/java-1.7.0-openjdk' '/usr/lib/jvm/java-7-openjdk') + local OPENJAVA7_HOME_CANDIDATES + JAVA8_HOME_CANDIDATES=('/usr/java/jdk1.8' '/usr/java/jre1.8' '/usr/lib/jvm/j2sdk1.8-oracle' '/usr/lib/jvm/j2sdk1.8-oracle/jre' '/usr/lib/jvm/java-8-oracle') + local JAVA8_HOME_CANDIDATES + OPENJAVA8_HOME_CANDIDATES=('/usr/lib/jvm/java-1.8.0-openjdk' '/usr/lib/jvm/java-8-openjdk' '/usr/lib64/jvm/java-1.8.0-openjdk' '/usr/lib64/jvm/java-8-openjdk') + local OPENJAVA8_HOME_CANDIDATES + MISCJAVA_HOME_CANDIDATES=('/Library/Java/Home' '/usr/java/default' '/usr/lib/jvm/default-java' '/usr/lib/jvm/java-openjdk' '/usr/lib/jvm/jre-openjdk') + local MISCJAVA_HOME_CANDIDATES + case ${BIGTOP_JAVA_MAJOR} in + JAVA_HOME_CANDIDATES=(${JAVA7_HOME_CANDIDATES[@]} ${JAVA8_HOME_CANDIDATES[@]} ${JAVA6_HOME_CANDIDATES[@]} ${MISCJAVA_HOME_CANDIDATES[@]} ${OPENJAVA7_HOME_CANDIDATES[@]} ${OPENJAVA8_HOME_CANDIDATES[@]} ${OPENJAVA6_HOME_CANDIDATES[@]}) + '[' -z /app/bds/java ']' + verify_java_home + '[' -z /app/bds/java ']' + echo JAVA_HOME=/app/bds/java + source_parcel_environment + '[' '!' -z '' ']' + JAVA=/app/bds/java/bin/java + [[ -z /usr/share/cmf ]] + MGMT_CLASSPATH='/usr/share/cmf/lib/*' + JDBC_JARS=/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar + MGMT_CLASSPATH='/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' ++ pwd + MGMT_CLASSPATH='/run/cloudera-scm-agent/process/572-MGMT.NAVIGATOR-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' + MGMT_JAVA_OPTS='-Djava.net.preferIPv4Stack=true ' + exec /app/bds/java/bin/java -Djava.net.preferIPv4Stack=true -Djava.security.egd=file:///dev/urandom -cp '/run/cloudera-scm-agent/process/572-MGMT.NAVIGATOR-test-db-connection:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-42.1.4.jre7.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/cmf/lib/*' com.cloudera.enterprise.dbutil.DbCommandExecutor db.properties Picked up JAVA_TOOL_OPTIONS: -Doracle.jdbc.thinLogonCapability=o3 -Djava.security.krb5.conf=/etc/krb5_bds.conf     root@abc% cat stdout.log Thu Jun 20 12:25:58 BST 2019 Current working directory: /run/cloudera-scm-agent/process/572-MGMT.NAVIGATOR-test-db-connection Launching one-off process: /usr/lib64/cmf/service/dbconnection/test_db_connection.sh db.properties Thu Jun 20 12:25:58 BST 2019 JAVA_HOME=/app/bds/java [ main] DbCommandExecutor INFO Successfully connected to database. ... View more

Hello Team, I already have balanced cluster where all datanodes have same space utilization. 6*4TB storage across all datanodes and all are 80% utilized. Since we do not have extra node to add for now we are adding extra strorage of 6*4TB across all datanodes. My question is: 1. Do i need to re-balance cluster? 2. If yes Do i still need to run balancer with default threshold of 10% or any else? - Vijay Mishra ... View more