Member since
02-10-2016
39
Posts
17
Kudos Received
0
Solutions
09-29-2016
01:55 PM
I was not able to solve the issue and finally decide to reinstall ambari that remove the problem. I didn't face the same kind of issue since then.
... View more
09-29-2016
01:54 PM
Sorry, I didn't see your answer sooner. I wasn't able to solve the issue and finally decide to reinstall ambari, that remove this issue. I didn't face the same kind of issue since then.
... View more
08-08-2016
06:55 PM
Hi, @rkovacs I don't have the problem when I downgrade to the latest version 1.2.2. So I think it is related to the last master version. Here is the complete error logs. I am going to submit an issue about that on github too. cloudbreak_1 | com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:86) ~[cloud-arm-1.2.5.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:46) ~[cloud-reactor-1.2.5.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:21) ~[cloud-reactor-1.2.5.jar!/:na]
cloudbreak_1 | at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source) ~[na:na]
cloudbreak_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
cloudbreak_1 | at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
cloudbreak_1 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at com.sun.proxy.$Proxy199.accept(Unknown Source) ~[na:na]
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160) [reactor-core-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74) [reactor-core-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
cloudbreak_1 | at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
cloudbreak_1 | Caused by: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateFromHttpStatus(StorageException.java:175) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:94) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:717) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:328) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:304) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer$createIfNotExists.call(Unknown Source) ~[na:na]
cloudbreak_1 | at com.sequenceiq.cloud.azure.client.AzureRMClient.createContainerInStorage(AzureRMClient.groovy:270) ~[azure-rest-client-0.1.56.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:79) ~[cloud-arm-1.2.5.jar!/:na]
cloudbreak_1 | ... 25 common frames omitted
Thank you for your help.
... View more
06-23-2016
01:15 PM
@Jitendra Yadav I increased the number of partition and it changed nothing. Each task take around 25 second Changing the number of executors didn't change anything either
... View more
06-23-2016
09:52 AM
Hello, I have a simple job, executing with spark-shell spark-shell --master yarn-client --num-executors 4 --executor-memory 4G val rdd = sc.textFile("some file")
rdd.count() File size : 20G, already on HDFS 141 blocks of 128M Cluster : 4 node of 7G The count job took ~20min. I feel that it is not a normal time. Should it be quicker ? The number of partitions : It is distributed : and the time for each task : The file is CSV file. I don't understand why the job take that long ? Is that a expected or is there a issue somewhere ? Regards
... View more
Labels:
06-21-2016
03:01 PM
Hello, In Ambari main page, I noticed that it is writtent that Yarn only have 7G RAM available : But I have a 4 nodes cluster with at least 11.5G Ram available Here is my Yarn configurations : Why Yarn doesn't have 28G RAM available ? Regards.
... View more
Labels:
05-17-2016
01:06 PM
Hi, @rkovacs I don't have the problem when I downgrade to the latest version 1.2.2. So I think it is related to the last master version. Here is the complete error logs. I am going to submit an issue about that on github too. cloudbreak_1 | com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:86) ~[cloud-arm-1.2.5.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:46) ~[cloud-reactor-1.2.5.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.handler.PrepareImageHandler.accept(PrepareImageHandler.java:21) ~[cloud-reactor-1.2.5.jar!/:na]
cloudbreak_1 | at sun.reflect.GeneratedMethodAccessor250.invoke(Unknown Source) ~[na:na]
cloudbreak_1 | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]
cloudbreak_1 | at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]
cloudbreak_1 | at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208) ~[spring-aop-4.2.4.RELEASE.jar!/:4.2.4.RELEASE]
cloudbreak_1 | at com.sun.proxy.$Proxy199.accept(Unknown Source) ~[na:na]
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:317) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus$3.accept(EventBus.java:310) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.routing.ConsumerFilteringRouter.route(ConsumerFilteringRouter.java:72) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.routing.TraceableDelegatingRouter.route(TraceableDelegatingRouter.java:51) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:591) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.bus.EventBus.accept(EventBus.java:63) [reactor-bus-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.core.dispatch.AbstractLifecycleDispatcher.route(AbstractLifecycleDispatcher.java:160) [reactor-core-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at reactor.core.dispatch.MultiThreadDispatcher$MultiThreadTask.run(MultiThreadDispatcher.java:74) [reactor-core-2.0.7.RELEASE.jar!/:na]
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
cloudbreak_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
cloudbreak_1 | at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
cloudbreak_1 | Caused by: com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateFromHttpStatus(StorageException.java:175) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:94) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.exists(CloudBlobContainer.java:717) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:328) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer.createIfNotExists(CloudBlobContainer.java:304) ~[azure-storage-4.0.0.jar!/:na]
cloudbreak_1 | at com.microsoft.azure.storage.blob.CloudBlobContainer$createIfNotExists.call(Unknown Source) ~[na:na]
cloudbreak_1 | at com.sequenceiq.cloud.azure.client.AzureRMClient.createContainerInStorage(AzureRMClient.groovy:270) ~[azure-rest-client-0.1.56.jar!/:na]
cloudbreak_1 | at com.sequenceiq.cloudbreak.cloud.arm.ArmSetup.prepareImage(ArmSetup.java:79) ~[cloud-arm-1.2.5.jar!/:na]
cloudbreak_1 | ... 25 common frames omitted
Thank you for your help.
... View more
05-13-2016
02:15 PM
Hi everyone, I have a cloudbreak instance running on a vm installed with cloudbreak-deployer and I can't create a cluster on Azure with it. Note that this is not a probleme on Azure side because I succefully create a cluster with the host version of cloudbreak (with the same credential) (https://cloudbreak.sequenceiq.com/). Here is the message error : com.microsoft.azure.storage.StorageException: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. The cloudbreak on my vm : local version:1.3.0-master-21fe5db
... View more
Labels:
04-25-2016
12:19 PM
I just noticed that after some time I get this JS error :
... View more
04-25-2016
12:11 PM
@Marton Sereg I pulled the lastest rc container(I did checked it was the right version with docker ps) but I still have the same issue :
... View more
04-25-2016
08:08 AM
@Marton Sereg Sorry to bother you again but I have a new issue related to the post. Cloudbreak api is running. It returns json as expected but I can't access the api with the UI. It seems there is something missing again.
... View more
04-22-2016
09:35 AM
@Marton Sereg The container was not running. I restart it like you said and it works. Thank you for your help !
... View more
04-22-2016
09:16 AM
Yes, but I have nothing listening to port 3000. And I don't see the ui container (uluwatu ?) with docker -ps
... View more
04-22-2016
08:51 AM
@Marton Sereg Thank you, the process is indeed running but I have a new issue : 2016-04-22 09:44:02,208 [main] start:162 INFO o.s.b.c.e.t.TomcatEmbeddedServletContainer - [owner:spring] [type:springLog] [id:] [name:] Tomcat started on port(s): 9091 (http)
2016-04-22 09:44:02,237 [main] logStarted:57 INFO c.s.c.CloudbreakApplication - [owner:spring] [type:springLog] [id:] [name:] Started CloudbreakApplication in 81.866 seconds (JVM running for 83.898) So it seems that the application listens port 9091. But in my browser I found :
... View more
04-21-2016
01:26 PM
It was indeed a problem with the host that I gave. Th ip adress of the posgre container was different. I don't have any errors when I run gradle bootRun, but it is stuck there: 2016-04-21 14:14:04,689 [main] logStarted:57 INFO c.s.c.CloudbreakApplication - [owner:spring] [type:springLog] [id:] [name:] Started CloudbreakApplication in 56.011 seconds (JVM running for 57.589)
2016-04-21 14:14:56,639 [MessageBroker-1] run:113 INFO o.s.w.s.c.WebSocketMessageBrokerStats - [owner:spring] [type:springLog] [id:] [name:] WebSocketSession[0 current WS(0)-HttpStream(0)-HttpPoll(0), 0 total, 0 closed abnormally (0 connect failure, 0 send limit, 0 transport error)], stompSubProtocol[processed CONNECT(0)-CONNECTED(0)-DISCONNECT(0)], stompBrokerRelay[null], inboundChannel[pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], outboundChannelpool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0], sockJsScheduler[pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]
> Building 98% > :core:bootRun
It's been 10 min and nothing change.
... View more
04-21-2016
12:41 PM
I believed they are correct : cb.cert.dir=/root/cbd-local/certs
cb.client.id=cloudbreak
cb.client.secret=cbsecret2015
cb.db.port.5432.tcp.addr=10.0.0.30
cb.db.port.5432.tcp.port=5432
spring.cloud.consul.host=10.0.0.30
cb.identity.server.url=http://10.0.0.30:8089
server.port=9091
and my ip is : inet 10.0.0.30 Is there something I am missing ?
... View more
04-21-2016
11:57 AM
I face a new error : 2016-04-21 12:57:23,367 [main] locate:90 WARN o.s.c.c.c.ConsulPropertySourceLocator - [owner:spring] [type:springLog] [id:] [name:] Unable to load consul config from config/null/
com.ecwid.consul.transport.TransportException: java.net.ConnectException: Connection refused
at com.ecwid.consul.transport.DefaultHttpTransport.executeRequest(DefaultHttpTransport.java:123) ~[consul-api-1.10.jar!/:na]
at com.ecwid.consul.transport.DefaultHttpTransport.makeGetRequest(DefaultHttpTransport.java:82) ~[consul-api-1.10.jar!/:na]
at com.ecwid.consul.v1.ConsulRawClient.makeGetRequest(ConsulRawClient.java:85) ~[consul-api-1.10.jar!/:na]
at com.ecwid.consul.v1.kv.KeyValueConsulClient.getKVValues(KeyValueConsulClient.java:135) ~[consul-api-1.10.jar!/:na]
at com.ecwid.consul.v1.ConsulClient.getKVValues(ConsulClient.java:364) ~[consul-api-1.10.jar!/:na]
at org.springframework.cloud.consul.config.ConsulPropertySource.init(ConsulPropertySource.java:61) ~[spring-cloud-consul-config-1.0.0.M6-v3.jar!/:1.0.0.M6-v3]
at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.locate(ConsulPropertySourceLocator.java:84) ~[spring-cloud-consul-config-1.0.0.M6-v3.jar!/:1.0.0.M6-v3]
at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:89) [spring-cloud-context-1.1.0.BUILD-SNAPSHOT.jar!/:1.1.0.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:638) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.doRun(SpringApplication.java:338) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:305) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1124) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1113) [spring-boot-1.3.1.RELEASE.jar!/:1.3.1.RELEASE]
at com.sequenceiq.cloudbreak.CloudbreakApplication.main(CloudbreakApplication.java:20) [cloudbreak.jar!/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_77]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_77]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_77]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [cloudbreak.jar!/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_77]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_77]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_77]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_77]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_77]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_77]
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:220) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164) ~[httpclient-4.5.1.jar!/:4.5.1]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139) ~[httpclient-4.5.1.jar!/:4.5.1]
at com.ecwid.consul.transport.DefaultHttpTransport.executeRequest(DefaultHttpTransport.java:107) ~[consul-api-1.10.jar!/:na]
... View more
04-21-2016
11:38 AM
Hello, I tried to build and run cloudbreak from the github project. But I face this error when I run ./gradlew bootRun -Dspring.config.location=application.properties --stacktrace * Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':autoscale:bootRun'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:306)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
at org.gradle.execution.taskgraph.ParallelTaskPlanExecutor.process(ParallelTaskPlanExecutor.java:45)
at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:169)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
at org.gradle.launcher.Main.doAction(Main.java:33)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:58)
Caused by: org.gradle.process.internal.ExecException: Process 'command '/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64/bin/java'' finished with non-zero exit value 1
at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:365)
at org.gradle.process.internal.DefaultJavaExecAction.execute(DefaultJavaExecAction.java:31)
at org.gradle.api.tasks.JavaExec.exec(JavaExec.java:60)
at org.springframework.boot.gradle.run.BootRunTask.exec(BootRunTask.java:63)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
... 47 more
I didn't manage to find a solution to fix this problem. for information : JAVA_HOME= /lib/jvm/java/
... View more
Labels:
04-13-2016
09:57 AM
1 Kudo
Hi, Is there a SPI to run Cloudbreak with an azure pack cloud platform(private cloud) ? I saw that Cloudbreak support microsoft azure but does it also support azure pack ?
... View more
Labels:
03-18-2016
10:57 AM
I still have the same problem,
Is it possible that the problem is : Connecting to https://dl-master:8440/ca ERROR 2016-03-16 10:18:31,414 NetUtil.py:77 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
I don't have any certificate with amabari-agent. But I didn't have any problem before.
... View more
03-16-2016
12:23 PM
I don't know if that the status : what would be the next step to solve the problem ?
... View more
03-16-2016
09:30 AM
Here is the ambari-alert log : Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException: Call From dl-s01/10.0.0.5 to dl-master:8020 failed on connection exception: java.net.ConnectException: Connection refused Caused by: java.net.ConnectException: Call From dl-s01/10.0.0.5 to dl-master:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1431)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:596)
at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:554)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:508)
... 8 more Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:612)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:710)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
at org.apache.hadoop.ipc.Client.call(Client.java:1397)
... 28 more)
... View more
03-16-2016
09:20 AM
Agent log :
INFO 2016-03-16 10:18:31,247 NetUtil.py:59 - Connecting to https://dl-master:8440/ca
ERROR 2016-03-16 10:18:31,414 NetUtil.py:77 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590) ERROR 2016-03-16 10:18:31,414 NetUtil.py:78 - SSLError: Failed to connect. Please check openssl library versions.
Refer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details. WARNING 2016-03-16 10:18:31,417 NetUtil.py:105 - Server at https://bugzilla.redhat.com/show_bug.cgi?id=1022468 is not reachable, sleeping for 10 seconds... WARNING 2016-03-16 10:18:31,417 NetUtil.py:105 - Server at https://bugzilla.redhat.com/show_bug.cgi?id=1022468 is not reachable, sleeping for 10 seconds... Server log : Mostly this : 16 Mar 2016 10:21:38,909 INFO [qtp-client-4711] MetricsPropertyProvider:518 - METRICS_COLLECTOR host is not live. Skip populating resources with metrics.
16 Mar 2016 10:21:38,910 INFO [qtp-client-4711] MetricsPropertyProvider:518 - METRICS_COLLECTOR host is not live. Skip populating resources with metrics.
... View more
03-16-2016
09:18 AM
So I did the operations, but nothing changes. In my ambari-agent log it shows WARNING 2016-03-16 10:18:00,938 NetUtil.py:105 - Server at https://dl-master:8440 is not reachable, sleeping for 10 seconds... The cluster have 4 nodes and only loose heartbeat on one node.
... View more
03-15-2016
04:45 PM
I have version2.1.1 ps -ef | grep kPT gives nothing : root 8364 8276 0 17:43 pts/1 00:00:00 grep kPT ambari-agent status gives : Found ambari-agent PID: 2123 ambari-agent running. Agent PID at: /var/run/ambari-agent/ambari-agent.pid Agent out at: /var/log/ambari-agent/ambari-agent.out Agent log at: /var/log/ambari-agent/ambari-agent.log
... View more
03-15-2016
04:17 PM
3 Kudos
Hello, I don't really know why but I lost all heartbeats on the main node of my cluster. Do you know how I can solve this problem ? I already try to reboot manually the node.
... View more
Labels:
02-26-2016
10:21 AM
2 Kudos
Hello, I don't have the lineage for certain of my process which succeeded. But I did have the lineage for the same process earlier this week : I didn't manage to find an explanation for this.
... View more
Labels:
02-25-2016
01:03 PM
1 Kudo
Hello, I am searching a way to do this : On HDFS, new datas arrived on a certain folder, and in the datas, there could be metadatas like names, emails, that are defined, and I would like a way to use these metadas to class my datas. I know that you can define tags on Falcon, but it is not automatically. Is there a way to automatically use the metadatas that are in a file, as to transfer them to falcon or another tools ? The goal is to use a Search tool (like there is in falcon).
... View more
Labels:
02-19-2016
09:26 AM
1 Kudo
@Benjamin Leonhardi I still have the same error even if when I change the data path : org.apache.falcon.FalconException: Couldn't evict feed from fileSystem And the feed is : <feed xmlns='uri:falcon:feed:0.1' name='Depot'>
<tags>projet=103,data=raw</tags>
<frequency>hours(1)</frequency>
<timezone>UTC</timezone>
<clusters>
<cluster name='primaryCluster' type='source'>
<validity start='2016-02-18T10:11Z' end='2016-02-19T10:11Z'/>
<retention limit='hours(4)' action='delete'/>
<locations>
<location type='data'>
</location>
<location type='stats'>
</location>
<location type='meta'>
</location>
</locations>
</cluster>
</clusters>
<locations>
<location type='data' path='/user/hdfs/falcon/DepotProject103/${YEAR}-${MONTH}-${DAY}-${HOUR}'>
</location>
<location type='stats' path='/'>
</location>
<location type='meta' path='/'>
</location>
</locations>
<ACL owner='ambari-qa' group='users' permission='0x755'/>
<schema location='/none' provider='/none'/>
<properties>
<property name='jobPriority' value='NORMAL'>
</property>
</properties>
</feed>
I think I did it right this time ? Is there something I am missing again ?
... View more