About arthur_grevin

arthur_grevin · ‎09-29-2016

I was not able to solve the issue and finally decide to reinstall ambari that remove the problem. I didn't face the same kind of issue since then.

arthur_grevin · ‎09-29-2016

Sorry, I didn't see your answer sooner. I wasn't able to solve the issue and finally decide to reinstall ambari, that remove this issue. I didn't face the same kind of issue since then.

arthur_grevin · ‎08-08-2016

Hi, @rkovacs I don't have the problem when I downgrade to the latest version 1.2.2. So I think it is related to the last master version. Here is the complete error logs. I am going to submit an issue about that on github too. cloudbreak_1 | com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:86) ~[cloud-arm-1.2.5.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:46) ~[cloud-reactor-1.2.5.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:21) ~[cloud-reactor-1.2.5.jar!/:na] cloudbreak_1 | at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source) ~[na:na] cloudbreak_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79] cloudbreak_1 | at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79] cloudbreak_1 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at com.sun.proxy.$Proxy199.accept(Unknown Source) ~[na:na] cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160) [reactor-core-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74) [reactor-core-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79] cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79] cloudbreak_1 | at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] cloudbreak_1 | Caused by: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateFromHttpStatus(StorageException.java:175) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:94) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:717) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:328) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:304) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer$createIfNotExists.call(Unknown Source) ~[na:na] cloudbreak_1 | at com.sequenceiq.cloud.azure.client.AzureRMClient.createContainerInStorage(AzureRMClient.groovy:270) ~[azure-rest-client-0.1.56.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:79) ~[cloud-arm-1.2.5.jar!/:na] cloudbreak_1 | ... 25 common frames omitted Thank you for your help.

arthur_grevin · ‎06-23-2016

@Jitendra Yadav I increased the number of partition and it changed nothing. Each task take around 25 second Changing the number of executors didn't change anything either

arthur_grevin · ‎06-23-2016

Hello, I have a simple job, executing with spark-shell spark-shell --master yarn-client --num-executors 4 --executor-memory 4G val rdd = sc.textFile("some file") rdd.count() File size : 20G, already on HDFS 141 blocks of 128M Cluster : 4 node of 7G The count job took ~20min. I feel that it is not a normal time. Should it be quicker ? The number of partitions : It is distributed : and the time for each task : The file is CSV file. I don't understand why the job take that long ? Is that a expected or is there a issue somewhere ? Regards

arthur_grevin · ‎06-21-2016

Hello, In Ambari main page, I noticed that it is writtent that Yarn only have 7G RAM available : But I have a 4 nodes cluster with at least 11.5G Ram available Here is my Yarn configurations : Why Yarn doesn't have 28G RAM available ? Regards.

arthur_grevin · ‎05-17-2016

Hi, @rkovacs I don't have the problem when I downgrade to the latest version 1.2.2. So I think it is related to the last master version. Here is the complete error logs. I am going to submit an issue about that on github too. cloudbreak_1 | com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:86) ~[cloud-arm-1.2.5.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:46) ~[cloud-reactor-1.2.5.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:21) ~[cloud-reactor-1.2.5.jar!/:na] cloudbreak_1 | at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source) ~[na:na] cloudbreak_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79] cloudbreak_1 | at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79] cloudbreak_1 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE] cloudbreak_1 | at com.sun.proxy.$Proxy199.accept(Unknown Source) ~[na:na] cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63) [reactor-bus-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160) [reactor-core-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74) [reactor-core-2.0.7.RELEASE.jar!/:na] cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79] cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79] cloudbreak_1 | at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] cloudbreak_1 | Caused by: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateFromHttpStatus(StorageException.java:175) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:94) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:717) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:328) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:304) ~[azure-storage-4.0.0.jar!/:na] cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer$createIfNotExists.call(Unknown Source) ~[na:na] cloudbreak_1 | at com.sequenceiq.cloud.azure.client.AzureRMClient.createContainerInStorage(AzureRMClient.groovy:270) ~[azure-rest-client-0.1.56.jar!/:na] cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:79) ~[cloud-arm-1.2.5.jar!/:na] cloudbreak_1 | ... 25 common frames omitted Thank you for your help.

arthur_grevin · ‎05-13-2016

Hi everyone, I have a cloudbreak instance running on a vm installed with cloudbreak-deployer and I can't create a cluster on Azure with it. Note that this is not a probleme on Azure side because I succefully create a cluster with the host version of cloudbreak (with the same credential) (https://cloudbreak.sequenceiq.com/). Here is the message error : com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. The cloudbreak on my vm : local version:1.3.0-master-21fe5db

arthur_grevin · ‎04-25-2016

I just noticed that after some time I get this JS error :

arthur_grevin · ‎04-25-2016

@Marton Sereg I pulled the lastest rc container(I did checked it was the right version with docker ps) but I still have the same issue :

arthur_grevin · ‎04-25-2016

@Marton Sereg Sorry to bother you again but I have a new issue related to the post. Cloudbreak api is running. It returns json as expected but I can't access the api with the UI. It seems there is something missing again.

arthur_grevin · ‎04-22-2016

@Marton Sereg The container was not running. I restart it like you said and it works. Thank you for your help !

arthur_grevin · ‎04-22-2016

Yes, but I have nothing listening to port 3000. And I don't see the ui container (uluwatu ?) with docker -ps

arthur_grevin · ‎04-22-2016

@Marton Sereg Thank you, the process is indeed running but I have a new issue : 2016-04-22 09:44:02,208 [main] start:162 INFO o.s.b.c.e.t.TomcatEmbeddedServletContainer - [owner:spring] [type:springLog] [id:] [name:] Tomcat started on port(s): 9091 (http) 2016-04-22 09:44:02,237 [main] logStarted:57 INFO c.s.c.CloudbreakApplication - [owner:spring] [type:springLog] [id:] [name:] Started CloudbreakApplication in 81.866 seconds (JVM running for 83.898) So it seems that the application listens port 9091. But in my browser I found :

arthur_grevin · ‎04-21-2016

It was indeed a problem with the host that I gave. Th ip adress of the posgre container was different. I don't have any errors when I run gradle bootRun, but it is stuck there: 2016-04-21 14:14:04,689 [main] logStarted:57 INFO c.s.c.CloudbreakApplication - [owner:spring] [type:springLog] [id:] [name:] Started CloudbreakApplication in 56.011 seconds (JVM running for 57.589) 2016-04-21 14:14:56,639 [MessageBroker-1] run:113 INFO o.s.w.s.c.WebSocketMessageBrokerStats - [owner:spring] [type:springLog] [id:] [name:] WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(0)-CONNECTED(0)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannelpool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0] > Building 98% > :core:bootRun It's been 10 min and nothing change.

arthur_grevin · ‎04-21-2016

I believed they are correct : cb.cert.dir=/root/cbd-local/certs cb.client.id=cloudbreak cb.client.secret=cbsecret2015 cb.db.port.5432.tcp.addr=10.0.0.30 cb.db.port.5432.tcp.port=5432 spring.cloud.consul.host=10.0.0.30 cb.identity.server.url=http://10.0.0.30:8089 server.port=9091 and my ip is : inet 10.0.0.30 Is there something I am missing ?

arthur_grevin · ‎04-21-2016

I face a new error : 2016-04-21 12:57:23,367 [main] locate:90 WARN o.s.c.c.c.ConsulPropertySourceLocator - [owner:spring] [type:springLog] [id:] [name:] Unable to load consul config from config/null/ com.ecwid.consul.transport.TransportException: java.net.ConnectException: Connection refused at com.ecwid.consul.transport.DefaultHttpTransport.executeRequest(DefaultHttpTransport.java:123) ~[consul-api-1.10.jar!/:na] at com.ecwid.consul.transport.DefaultHttpTransport.makeGetRequest(DefaultHttpTransport.java:82) ~[consul-api-1.10.jar!/:na] at com.ecwid.consul.v1.ConsulRawClient.makeGetRequest(ConsulRawClient.java:85) ~[consul-api-1.10.jar!/:na] at com.ecwid.consul.v1.kv.KeyValueConsulClient.getKVValues(KeyValueConsulClient.java:135) ~[consul-api-1.10.jar!/:na] at com.ecwid.consul.v1.ConsulClient.getKVValues(ConsulClient.java:364) ~[consul-api-1.10.jar!/:na] at org.springframework.cloud.consul.config.ConsulPropertySource.init(ConsulPropertySource.java:61) ~[spring-cloud-consul-config-1.0.0.M6-v3.jar!/:1.0.0.M6-v3] at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.locate(ConsulPropertySourceLocator.java:84) ~[spring-cloud-consul-config-1.0.0.M6-v3.jar!/:1.0.0.M6-v3] at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:89) [spring-cloud-context-1.1.0.BUILD-SNAPSHOT.jar!/:1.1.0.BUILD-SNAPSHOT] at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:638) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE] at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:338) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE] at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE] at com.sequenceiq.cloudbreak.CloudbreakApplication.main(CloudbreakApplication.java:20) [cloudbreak.jar!/:na] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77] at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [cloudbreak.jar!/:na] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77] Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_77] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_77] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_77] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_77] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_77] at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_77] at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:220) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164) ~[httpclient-4.5.1.jar!/:4.5.1] at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139) ~[httpclient-4.5.1.jar!/:4.5.1] at com.ecwid.consul.transport.DefaultHttpTransport.executeRequest(DefaultHttpTransport.java:107) ~[consul-api-1.10.jar!/:na]

arthur_grevin · ‎04-21-2016

Hello, I tried to build and run cloudbreak from the github project. But I face this error when I run ./gradlew bootRun -Dspring.config.location=application.properties --stacktrace * Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':autoscale:bootRun'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:306) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63) at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51) at org.gradle.execution.taskgraph.ParallelTaskPlanExecutor.process(ParallelTaskPlanExecutor.java:45) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88) at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62) at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23) at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68) at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62) at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55) at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149) at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106) at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86) at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80) at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33) at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24) at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36) at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26) at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51) at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:169) at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237) at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210) at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35) at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24) at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206) at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169) at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33) at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22) at org.gradle.launcher.Main.doAction(Main.java:33) at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45) at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54) at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35) at org.gradle.launcher.GradleMain.main(GradleMain.java:23) at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30) at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127) at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:58) Caused by: org.gradle.process.internal.ExecException: Process 'command '/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64/bin/java'' finished with non-zero exit value 1 at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:365) at org.gradle.process.internal.DefaultJavaExecAction.execute(DefaultJavaExecAction.java:31) at org.gradle.api.tasks.JavaExec.exec(JavaExec.java:60) at org.springframework.boot.gradle.run.BootRunTask.exec(BootRunTask.java:63) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63) at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218) at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211) at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61) ... 47 more I didn't manage to find a solution to fix this problem. for information : JAVA_HOME= /lib/jvm/java/

arthur_grevin · ‎04-13-2016

Hi, Is there a SPI to run Cloudbreak with an azure pack cloud platform(private cloud) ? I saw that Cloudbreak support microsoft azure but does it also support azure pack ?

arthur_grevin · ‎03-18-2016

I still have the same problem, Is it possible that the problem is : Connecting to https://dl-master:8440/ca ERROR 2016-03-16 10:18:31,414 NetUtil.py:77 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) I don't have any certificate with amabari-agent. But I didn't have any problem before.

arthur_grevin · ‎03-16-2016

I don't know if that the status : what would be the next step to solve the problem ?

arthur_grevin · ‎03-16-2016

Here is the ambari-alert log : Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException: Call From dl-s01/10.0.0.5 to dl-master:8020 failed on connection exception: java.net.ConnectException: Connection refused Caused by: java.net.ConnectException: Call From dl-s01/10.0.0.5 to dl-master:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732) at org.apache.hadoop.ipc.Client.call(Client.java:1431) at org.apache.hadoop.ipc.Client.call(Client.java:1358) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:596) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:554) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508) ... 8 more Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710) at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493) at org.apache.hadoop.ipc.Client.call(Client.java:1397) ... 28 more)

arthur_grevin · ‎03-16-2016

Agent log : INFO 2016-03-16 10:18:31,247 NetUtil.py:59 - Connecting to https://dl-master:8440/ca ERROR 2016-03-16 10:18:31,414 NetUtil.py:77 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) ERROR 2016-03-16 10:18:31,414 NetUtil.py:78 - SSLError: Failed to connect. Please check openssl library versions. Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details. WARNING 2016-03-16 10:18:31,417 NetUtil.py:105 - Server at https://bugzilla.redhat.com/show_bug.cgi?id=1022468 is not reachable, sleeping for 10 seconds... WARNING 2016-03-16 10:18:31,417 NetUtil.py:105 - Server at https://bugzilla.redhat.com/show_bug.cgi?id=1022468 is not reachable, sleeping for 10 seconds... Server log : Mostly this : 16 Mar 2016 10:21:38,909 INFO [qtp-client-4711] MetricsPropertyProvider:518 - METRICS_COLLECTOR host is not live. Skip populating resources with metrics. 16 Mar 2016 10:21:38,910 INFO [qtp-client-4711] MetricsPropertyProvider:518 - METRICS_COLLECTOR host is not live. Skip populating resources with metrics.

arthur_grevin · ‎03-16-2016

So I did the operations, but nothing changes. In my ambari-agent log it shows WARNING 2016-03-16 10:18:00,938 NetUtil.py:105 - Server at https://dl-master:8440 is not reachable, sleeping for 10 seconds... The cluster have 4 nodes and only loose heartbeat on one node.

arthur_grevin · ‎03-15-2016

I have version2.1.1 ps -ef | grep kPT gives nothing : root 8364 8276 0 17:43 pts/1 00:00:00 grep kPT ambari-agent status gives : Found ambari-agent PID: 2123 ambari-agent running. Agent PID at: /var/run/ambari-agent/ambari-agent.pid Agent out at: /var/log/ambari-agent/ambari-agent.out Agent log at: /var/log/ambari-agent/ambari-agent.log

arthur_grevin · ‎03-15-2016

Hello, I don't really know why but I lost all heartbeats on the main node of my cluster. Do you know how I can solve this problem ? I already try to reboot manually the node.

arthur_grevin · ‎02-26-2016

Hello, I don't have the lineage for certain of my process which succeeded. But I did have the lineage for the same process earlier this week : I didn't manage to find an explanation for this.

arthur_grevin · ‎02-26-2016

Thank you for your answer.

arthur_grevin · ‎02-25-2016

Hello, I am searching a way to do this : On HDFS, new datas arrived on a certain folder, and in the datas, there could be metadatas like names, emails, that are defined, and I would like a way to use these metadas to class my datas. I know that you can define tags on Falcon, but it is not automatically. Is there a way to automatically use the metadatas that are in a file, as to transfer them to falcon or another tools ? The goal is to use a Search tool (like there is in falcon).

arthur_grevin · ‎02-19-2016

@Benjamin Leonhardi I still have the same error even if when I change the data path : org.apache.falcon.FalconException: Couldn't evict feed from fileSystem And the feed is : <feed xmlns='uri:falcon:feed:0.1' name='Depot'> <tags>projet=103,data=raw</tags> <frequency>hours(1)</frequency> <timezone>UTC</timezone> <clusters> <cluster name='primaryCluster' type='source'> <validity start='2016-02-18T10:11Z' end='2016-02-19T10:11Z'/> <retention limit='hours(4)' action='delete'/> <locations> <location type='data'> </location> <location type='stats'> </location> <location type='meta'> </location> </locations> </cluster> </clusters> <locations> <location type='data' path='/user/hdfs/falcon/DepotProject103/${YEAR}-${MONTH}-${DAY}-${HOUR}'> </location> <location type='stats' path='/'> </location> <location type='meta' path='/'> </location> </locations> <ACL owner='ambari-qa' group='users' permission='0x755'/> <schema location='/none' provider='/none'/> <properties> <property name='jobPriority' value='NORMAL'> </property> </properties> </feed> I think I did it right this time ? Is there something I am missing again ?

Online	Offline
Last Visited	‎09-29-2016 01:54 PM

Date Registered	‎02-10-2016 09:38 AM
Last Visited	‎09-29-2016 01:54 PM
Total Messages Posted	39
Total Kudos Received	17

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Re: Cloudbreak fail install on Azure

Re: Spark performance

Spark performance

Yarn available memory

Re: Cloudbreak fail install on Azure

Cloudbreak fail install on Azure

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Re: Build Failed gradle bootrun Cloudbreak

Build Failed gradle bootrun Cloudbreak

Cloud Break and Azure Pack

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Re: Lost HeartBeats Ambari

Lost HeartBeats Ambari

Lineage Falcon missing

Re: Add metadata automatically

Add metadata automatically

Re: Feed : Unable to resolve pattern for feedPath