Created 09-20-2016 10:55 AM
Hi Mike:
How can I attached the /var/log/cloudera-director-server/application.log text file in this post, when I look I could not find any button or icon that I can attach text file.
I will use the "Insert Code" icon to insert the errors that I grep from /var/log/cloudera-director-server/application.log, and please let me know if you spot anything that may help me out.
[2016-09-20 16:09:53] ERROR [pipeline-thread-149] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:09:53] ERROR [pipeline-thread-150] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:09:58] ERROR [pipeline-thread-152] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:09:58] ERROR [pipeline-thread-151] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:02] ERROR [pipeline-thread-154] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:06] ERROR [pipeline-thread-149] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:06] ERROR [pipeline-thread-150] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:10] ERROR [pipeline-thread-152] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:11] ERROR [pipeline-thread-151] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:14] ERROR [pipeline-thread-157] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:14] ERROR [pipeline-thread-156] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:14] ERROR [pipeline-thread-158] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:14] ERROR [pipeline-thread-154] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:19] ERROR [pipeline-thread-149] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:19] ERROR [pipeline-thread-150] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:23] ERROR [pipeline-thread-152] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:23] ERROR [pipeline-thread-151] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:26] ERROR [pipeline-thread-153] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:26] ERROR [pipeline-thread-157] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:27] ERROR [pipeline-thread-156] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:27] ERROR [pipeline-thread-158] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:27] ERROR [pipeline-thread-154] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:27] ERROR [pipeline-thread-155] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:39] ERROR [pipeline-thread-153] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:39] ERROR [pipeline-thread-157] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:39] ERROR [pipeline-thread-156] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:40] ERROR [pipeline-thread-158] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:40] ERROR [pipeline-thread-155] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:41] ERROR [pipeline-thread-149] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:41] ERROR [pipeline-thread-150] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:46] ERROR [pipeline-thread-152] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:46] ERROR [pipeline-thread-151] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:50] ERROR [pipeline-thread-154] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:51] ERROR [pipeline-thread-153] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:10:53] ERROR [pipeline-thread-155] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:02] ERROR [pipeline-thread-157] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:02] ERROR [pipeline-thread-156] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:02] ERROR [pipeline-thread-158] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:14] ERROR [pipeline-thread-153] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:14] ERROR [pipeline-thread-149] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:14] ERROR [pipeline-thread-149] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00004-9934ae87-37b2-4b56-99c7-1ff793b5972c/child-00000-1e67b9d4-1fd9-49b8-801a-ccbfda53195d suspended due to failure [2016-09-20 16:11:14] ERROR [pipeline-thread-149] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00004-9934ae87-37b2-4b56-99c7-1ff793b5972c/child-00000-1e67b9d4-1fd9-49b8-801a-ccbfda53195d' failed [2016-09-20 16:11:14] ERROR [pipeline-thread-150] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:14] ERROR [pipeline-thread-150] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00002-e4fb4729-e05b-47d9-bafd-d9dddd09aed7/child-00000-8ba2490e-c661-4a53-8adf-57749d7e5607 suspended due to failure [2016-09-20 16:11:14] ERROR [pipeline-thread-150] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00002-e4fb4729-e05b-47d9-bafd-d9dddd09aed7/child-00000-8ba2490e-c661-4a53-8adf-57749d7e5607' failed [2016-09-20 16:11:14] ERROR [pipeline-thread-147] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:14] ERROR [pipeline-thread-147] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:14] ERROR [pipeline-thread-147] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00004-9934ae87-37b2-4b56-99c7-1ff793b5972c' failed [2016-09-20 16:11:14] INFO [pipeline-thread-147] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00004-9934ae87-37b2-4b56-99c7-1ff793b5972c': RUNNING -> ERROR [2016-09-20 16:11:15] ERROR [pipeline-thread-145] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:15] ERROR [pipeline-thread-145] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:15] ERROR [pipeline-thread-145] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00002-e4fb4729-e05b-47d9-bafd-d9dddd09aed7' failed [2016-09-20 16:11:15] INFO [pipeline-thread-145] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00002-e4fb4729-e05b-47d9-bafd-d9dddd09aed7': RUNNING -> ERROR [2016-09-20 16:11:15] ERROR [pipeline-thread-155] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:19] ERROR [pipeline-thread-152] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:19] ERROR [pipeline-thread-152] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3/child-00001-2f753dbf-e423-46ab-8a56-ec996960abee suspended due to failure [2016-09-20 16:11:19] ERROR [pipeline-thread-152] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3/child-00001-2f753dbf-e423-46ab-8a56-ec996960abee' failed [2016-09-20 16:11:19] ERROR [pipeline-thread-151] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:19] ERROR [pipeline-thread-151] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3/child-00000-cbf7dfb4-0d9c-43d1-a570-68e1a54a8b9d suspended due to failure [2016-09-20 16:11:19] ERROR [pipeline-thread-151] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3/child-00000-cbf7dfb4-0d9c-43d1-a570-68e1a54a8b9d' failed [2016-09-20 16:11:19] ERROR [pipeline-thread-146] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:19] ERROR [pipeline-thread-146] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:19] ERROR [pipeline-thread-146] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3' failed [2016-09-20 16:11:19] INFO [pipeline-thread-146] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00003-9820c23b-ff0a-4eb9-9a66-07beca609da3': RUNNING -> ERROR [2016-09-20 16:11:22] ERROR [pipeline-thread-154] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:22] ERROR [pipeline-thread-154] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00000-a8f30c95-d228-4fea-8cf2-6c25e1aad793/child-00000-589a57d1-4f70-4733-8e23-fb1ec7f48d40 suspended due to failure [2016-09-20 16:11:22] ERROR [pipeline-thread-154] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00000-a8f30c95-d228-4fea-8cf2-6c25e1aad793/child-00000-589a57d1-4f70-4733-8e23-fb1ec7f48d40' failed [2016-09-20 16:11:23] ERROR [pipeline-thread-143] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:23] ERROR [pipeline-thread-143] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:23] ERROR [pipeline-thread-143] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00000-a8f30c95-d228-4fea-8cf2-6c25e1aad793' failed [2016-09-20 16:11:23] INFO [pipeline-thread-143] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00000-a8f30c95-d228-4fea-8cf2-6c25e1aad793': RUNNING -> ERROR [2016-09-20 16:11:34] ERROR [pipeline-thread-157] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:34] ERROR [pipeline-thread-157] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00000-36b31044-6873-4fe5-a2b1-de8fbdb92cc2 suspended due to failure [2016-09-20 16:11:34] ERROR [pipeline-thread-157] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00000-36b31044-6873-4fe5-a2b1-de8fbdb92cc2' failed [2016-09-20 16:11:35] ERROR [pipeline-thread-156] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:35] ERROR [pipeline-thread-156] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00002-52f2769e-9f36-4e41-ae0a-75e09e00008e suspended due to failure [2016-09-20 16:11:35] ERROR [pipeline-thread-156] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00002-52f2769e-9f36-4e41-ae0a-75e09e00008e' failed [2016-09-20 16:11:35] ERROR [pipeline-thread-158] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:35] ERROR [pipeline-thread-158] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00001-14c1f6d1-6da6-4994-b810-dd6ddde9a569 suspended due to failure [2016-09-20 16:11:35] ERROR [pipeline-thread-158] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373/child-00001-14c1f6d1-6da6-4994-b810-dd6ddde9a569' failed [2016-09-20 16:11:35] ERROR [pipeline-thread-148] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:35] ERROR [pipeline-thread-148] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:35] ERROR [pipeline-thread-148] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373' failed [2016-09-20 16:11:35] INFO [pipeline-thread-148] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00005-ddd11da1-c30e-477a-b1ec-7b8a0b9bb373': RUNNING -> ERROR [2016-09-20 16:11:47] ERROR [pipeline-thread-153] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:47] ERROR [pipeline-thread-153] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616/child-00001-534e4a8d-b4c0-4424-b8b2-d0f08342947e suspended due to failure [2016-09-20 16:11:47] ERROR [pipeline-thread-153] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616/child-00001-534e4a8d-b4c0-4424-b8b2-d0f08342947e' failed [2016-09-20 16:11:48] ERROR [pipeline-thread-155] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:48] ERROR [pipeline-thread-155] - c.c.l.p.DatabasePipelineRunner: Pipeline 2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616/child-00000-82f63522-031c-4019-89f0-b3acf9efb701 suspended due to failure [2016-09-20 16:11:48] ERROR [pipeline-thread-155] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616/child-00000-82f63522-031c-4019-89f0-b3acf9efb701' failed [2016-09-20 16:11:48] ERROR [pipeline-thread-144] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:48] ERROR [pipeline-thread-144] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:48] ERROR [pipeline-thread-144] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616' failed [2016-09-20 16:11:48] INFO [pipeline-thread-144] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616': RUNNING -> ERROR [2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed [2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error [2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b' failed [2016-09-20 16:11:48] INFO [pipeline-thread-142] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b': RUNNING -> ERROR
Created 09-20-2016 10:58 AM
Mike:
I have been trying to attach the error from /var/log/cloudera-director-server/application.log and each time I post, the whole reply to you is disappears. Have I reach the threshold of this post, should I start a new post?
Created 09-20-2016 11:00 AM
Mike:
Please let me know how I can get the /var/log/cloudera-director-server/application.log file to you for investigation?
None of the icons or buttons in this post allows any attachment of text files...if you know of a way, please assists me.
Created 09-20-2016 11:03 AM
Mike:
I'm getting a lot of the following errors from /var/log/cloudera-director-server/application.log
[2016-09-20 16:11:48] INFO [pipeline-thread-144] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b/child-00001-6254aadd-e1bc-47cd-b0b1-7bd886dc4616': RUNNING -> ERROR
[2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.pipeline.util.PipelineRunner: Attempt to execute job failed
[2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.p.DatabasePipelineRunner: Encountered an unrecoverable error
[2016-09-20 16:11:48] ERROR [pipeline-thread-142] - c.c.l.p.DatabasePipelineRunner: Pipeline '2179431f-5d45-433c-90be-a1301f77149b' failed
[2016-09-20 16:11:48] INFO [pipeline-thread-142] - c.c.l.p.s.PipelineRepositoryService: Pipeline '2179431f-5d45-433c-90be-a1301f77149b': RUNNING -> ERROR
Created 09-20-2016 11:47 AM
Created 09-20-2016 01:57 PM
Hi Mike:
Done pasting all the lines from "/var/log/cloudera-director-server/application.log" and title of pastebin.com is the same.
Please download and I have made it expired in 1 day.
Thanks.
Beatrice
Created 09-20-2016 02:08 PM
Created 09-20-2016 02:10 PM
Mike:
Sorry it was my first time in pastebin.com... Let me know what you find, and if you need more information, post back a reply and I'll get the detail of what you need.
Created 09-21-2016 07:09 AM
Created on 09-21-2016 02:32 PM - edited 09-21-2016 02:33 PM
Mike:
I gave the AWS access key with admin permission, and rerun using "
cloudera-director bootstrap-remote aws.ha.cluster.conf --lp.remote.username=admin --lp.remote.password=xxxxxxx --lp.remote.hostAndPort=localhost:7189"
When I look at the
/var/log/cloudera-director-server/application.log - following are the errors. The cluster was not created, is there some bootstrap that I need for the config file to point to?
[2016-09-21 21:28:09] ERROR [pipeline-thread-171] - c.c.l.p.DatabasePipelineRunner: Pipeline 'dc944eea-3678-4c81-b56b-0cb9a8955312' failed
at com.cloudera.launchpad.pipeline.util.UnboundedWaitForAllPipelines$$EnhancerBySpringCGLIB$$eb8954a3
at com.cloudera.launchpad.pipeline.util.UnboundedParallelForEach:6
[2016-09-21 21:28:09] INFO [pipeline-thread-171] - c.c.l.p.s.PipelineRepositoryService: Pipeline 'dc944eea-3678-4c81-b56b-0cb9a8955312': RUNNING -> ERROR
[2016-09-21 21:28:09] INFO [pipeline-thread-171] - c.c.l.d.ClusterRepositoryService: Cluster 'cdh571nkha-new2': BOOTSTRAPPING -> BOOTSTRAP_FAILED
[2016-09-21 21:28:09] INFO [pipeline-thread-171] - c.c.l.m.r.DeploymentsListener: Cluster entity cdh571nkha-new2 changed.