Member since
08-20-2019
6
Posts
0
Kudos Received
0
Solutions
08-29-2019
08:43 AM
Director instance, - PUT /api/v11/environments/yliskevy-devlight-env/deployments/yliskevy-devlight-manager/clusters/yliskevy-devlight-cluster - - o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver: Resolving exception from handler [public void com.cloudera.launchpad.api.v11.ClustersResourceV11.update(java.lang.String,java.lang.String,java.lang.String,com.cloudera.launchpad.api.v11.model.cluster.ClusterTemplateV11) throws java.lang.InterruptedException]: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Can not construct instance of com.cloudera.launchpad.api.v11.model.cluster.ClusterTemplateV11, problem: null; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.cloudera.launchpad.api.v11.model.cluster.ClusterTemplateV11, problem: null
at [Source: java.io.PushbackInputStream@4919c70b; line: 32, column: 1]
... View more
08-29-2019
12:31 AM
Thanks Bill for your comment, I always got a response {
"timestamp": 1567007417642,
"status": 400,
"error": "Bad Request",
"exception": "org.springframework.http.converter.HttpMessageNotReadableException",
"message": "Bad Request",
"path": "/api/v11/environments/yliskevy-devlight-env/deployments/yliskevy-devlight-manager/clusters/yliskevy-devlight-cluster"
}
... View more
08-28-2019
07:23 AM
Hi All,
Please help me to find the solution to update cluster with Cloudera Director API v11.
I have tried to use PUT /api/v11/environments/{environment}/deployments/{deployment}/clusters/{cluster}
with body
{
"name": "yliskevy-devlight-cluster",
"productVersions": {
"CDH": "5.15.2"
},
"services": [
"HDFS",
"YARN",
"ZOOKEEPER",
"HIVE",
"IMPALA",
"OOZIE",
"HUE",
"AWS_S3"
],
"virtualInstanceGroups": {
"masters": {
"minCount": 0,
"name": "masters",
"virtualInstances": [
{
"id": "yliskevy-devlight-master-0",
"template": {
"name": "yliskevy-devlight-cluster-masters",
"type": "m5.xlarge",
"image": "ami-019fa65275c44d1fe"
}
}
]
}
}
}
but always got response
{
"name": "yliskevy-devlight-cluster",
"productVersions": {
"CDH": "5.15.2"
},
"services": [
"HDFS",
"YARN",
"ZOOKEEPER",
"HIVE",
"IMPALA",
"OOZIE",
"HUE",
"AWS_S3"
],
"virtualInstanceGroups": {
"masters": {
"minCount": 0,
"name": "masters",
"virtualInstances": [
{
"id": "yliskevy-devlight-master-0",
"template": {
"name": "yliskevy-devlight-cluster-masters",
"type": "m5.xlarge",
"image": "ami-019fa65275c45d1fe"
}
}
]
}
}
}
Logs from Director instance
WARN [qtp1868987089-19] cfeb9d95-c22b-4254-a3c8-4549e2ff5e7c PUT /api/v11/environments/yliskevy-devlight-env/deployments/yliskevy-devlight-manager/clusters/yliskevy-devlight-cluster - - o.s.w.s.m.s.DefaultHandlerExceptionResolver: Failed to read HTTP message: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Can not construct instance of com.cloudera.launchpad.api.v11.model.cluster.ClusterTemplateV11, problem: null; nested exception is com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of com.cloudera.launchpad.api.v11.model.cluster.ClusterTemplateV11, problem: null
If someone has correct template, please share it for me.
Thanks!
... View more
08-20-2019
07:02 AM
benhadoop, thank for your replays, This logs during the crashed (last logs). Changed to -Xmx4G didn't help.
... View more
08-20-2019
05:58 AM
2019-08-20 12:37:26,745 INFO 179986820@scm-web-19:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command First Run BasicCmdArgs{args=[CD-AWS_S3-Pznldbee, CD-ZOOKEEPER-OaAVutjk, CD-HDFS-CqmTPAuV, CD-KAFKA-ngsMquLo, CD-SENTRY-wFibPeTJ, CD-YARN-mrECVkcx, CD-HIVE-NAMKHwSU, CD-IMPALA-xDNLSCjr, CD-OOZIE-GplqKSro, CD-SQOOP-vUHtkubv, CD-HUE-VMbnNPaO]}.
2019-08-20 12:37:26,812 INFO 179986820@scm-web-19:com.cloudera.cmf.service.FirstRunCommand: Disabling Faster First Run due to presence of HDFS HA.
2019-08-20 12:37:26,812 INFO 179986820@scm-web-19:com.cloudera.cmf.service.FirstRunCommand: Faster First Run enabled: false. Start wait seconds: 20
2019-08-20 12:37:27,044 INFO 179986820@scm-web-19:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 8 steps in sequence
2019-08-20 12:37:27,044 INFO 179986820@scm-web-19:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in parallel
2019-08-20 12:37:27,044 INFO 179986820@scm-web-19:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Waiting for Cloudera Manager Agents to detect release: CDH 5.15.2.
2019-08-20 12:37:27,124 INFO 179986820@scm-web-19:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from 10.0.1.180
2019-08-20 12:37:27,138 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on cluster user-cluster
2019-08-20 12:37:27,669 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing command DeployClusterClientConfig BasicCmdArgs{args=[]}.
2019-08-20 12:37:27,669 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:27,669 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 5 steps in parallel
2019-08-20 12:37:27,669 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on service SENTRY-1
2019-08-20 12:37:27,727 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing service command deployClientConfig SvcCmdArgs{targetRoles=[DbRole{id=53, name=CD-S0265f753-SENTRY_SERVER-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal}], args=[]}. Service: DbService{id=12, name=CD-SENTRY-wFibPeTJ}
2019-08-20 12:37:27,728 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:27,728 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Generate and deploy client configuration.
2019-08-20 12:37:28,077 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on service HIVE-1
2019-08-20 12:37:28,536 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing service command deployClientConfig SvcCmdArgs{targetRoles=DbRole{id=9, name=CD-HIVE-NAMKHwSU-GATEWAY-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal},DbRole{id=10, name=CD-HIVE-NAMKHwSU-HIVEMETASTORE-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal},DbRole{id=18, name=CD-HIVE-NAMKHwSU-GATEWAY-323fb05f1245fb24dcd2a1d5c3e789e7, hostName=ip-10-0-1-50.ec2.internal},DbRole{id=21, name=CD-HIVE-NAMKHwSU-GATEWAY-85cddaaff129ad48e098c8bce79ab651, hostName=ip-10-0-3-22.ec2.internal},DbRole{id=17, name=CD-HIVE-NAMKHwSU-HIVESERVER2-481feee19833ca1500ab7b88936c8b73, hostName=ip-10-0-1-241.ec2.internal},DbRole{id=16, name=CD-HIVE-NAMKHwSU-HIVEMETASTORE-481feee19833ca1500ab7b88936c8b73, hostName=ip-10-0-1-241.ec2.internal},DbRole{id=11, name=CD-HIVE-NAMKHwSU-HIVESERVER2-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal},DbRole{id=13, name=CD-HIVE-NAMKHwSU-GATEWAY-94bb76bb3845d885b0b4b7aef47125e5, hostName=ip-10-0-1-140.ec2.internal},DbRole{id=20, name=CD-HIVE-NAMKHwSU-GATEWAY-737f61d86c70b41923ac33cfede3f583, hostName=ip-10-0-2-210.ec2.internal},DbRole{id=12, name=CD-HIVE-NAMKHwSU-GATEWAY-adf311de339f2c4fe6c7cc6b8d398b30, hostName=ip-10-0-1-113.ec2.internal}...(3 more roles), args=[]}. Service: DbService{id=3, name=CD-HIVE-NAMKHwSU}
2019-08-20 12:37:28,536 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:28,536 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Generate and deploy client configuration.
2019-08-20 12:37:29,212 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on service YARN-1
2019-08-20 12:37:29,402 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing service command deployClientConfig SvcCmdArgs{targetRoles=[DbRole{id=37, name=CD-YARN-mrECVkcx-NODEMANAGER-af56a93fe2cd7bb00ab8f433fbe46739, hostName=ip-10-0-2-161.ec2.internal}, DbRole{id=33, name=CD-Y70cff05d-RESOURCEMANAGER-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal}, DbRole{id=34, name=CD-YARN-mrECVkcx-JOBHISTORY-58cc2c2ba2290cbea8100cc7d79df953, hostName=ip-10-0-1-108.ec2.internal}, DbRole{id=38, name=CD-YARN-mrECVkcx-NODEMANAGER-85cddaaff129ad48e098c8bce79ab651, hostName=ip-10-0-3-22.ec2.internal}, DbRole{id=35, name=CD-YARN-mrECVkcx-NODEMANAGER-b31bec2d07093c32bee7c2f274aed6ba, hostName=ip-10-0-1-235.ec2.internal}, DbRole{id=36, name=CD-Y70cff05d-RESOURCEMANAGER-481feee19833ca1500ab7b88936c8b73, hostName=ip-10-0-1-241.ec2.internal}], args=[]}. Service: DbService{id=8, name=CD-YARN-mrECVkcx}
2019-08-20 12:37:29,402 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:29,402 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Generate and deploy client configuration.
2019-08-20 12:37:30,019 INFO metric-schema-updater:com.cloudera.cmon.components.MetricSchemaManager: Metric schema updated in PT8.326S.
2019-08-20 12:37:30,164 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on service HDFS-1
2019-08-20 12:37:30,303 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing service command deployClientConfig SvcCmdArgs{targetRoles=DbRole{id=40, name=CD-HDFS-CqmTPAuV-JOURNALNODE-adf311de339f2c4fe6c7cc6b8d398b30, hostName=ip-10-0-1-113.ec2.internal},DbRole{id=50, name=CD-HDFS-CqmTPAuV-DATANODE-85cddaaff129ad48e098c8bce79ab651, hostName=ip-10-0-3-22.ec2.internal},DbRole{id=41, name=CD-HDFS-CqmTPAuV-NAMENODE-adf311de339f2c4fe6c7cc6b8d398b30, hostName=ip-10-0-1-113.ec2.internal},DbRole{id=47, name=CD-H9881db27-FAILOVERCONTROLLER-737f61d86c70b41923ac33cfede3f583, hostName=ip-10-0-2-210.ec2.internal},DbRole{id=48, name=CD-HDFS-CqmTPAuV-JOURNALNODE-737f61d86c70b41923ac33cfede3f583, hostName=ip-10-0-2-210.ec2.internal},DbRole{id=44, name=CD-HDFS-CqmTPAuV-BALANCER-323fb05f1245fb24dcd2a1d5c3e789e7, hostName=ip-10-0-1-50.ec2.internal},DbRole{id=49, name=CD-HDFS-CqmTPAuV-NAMENODE-737f61d86c70b41923ac33cfede3f583, hostName=ip-10-0-2-210.ec2.internal},DbRole{id=42, name=CD-HDFS-CqmTPAuV-DATANODE-b31bec2d07093c32bee7c2f274aed6ba, hostName=ip-10-0-1-235.ec2.internal},DbRole{id=45, name=CD-HDFS-CqmTPAuV-HTTPFS-323fb05f1245fb24dcd2a1d5c3e789e7, hostName=ip-10-0-1-50.ec2.internal},DbRole{id=46, name=CD-HDFS-CqmTPAuV-DATANODE-af56a93fe2cd7bb00ab8f433fbe46739, hostName=ip-10-0-2-161.ec2.internal}...(2 more roles), args=[]}. Service: DbService{id=9, name=CD-HDFS-CqmTPAuV}
2019-08-20 12:37:30,309 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:30,309 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Generate and deploy client configuration.
2019-08-20 12:37:30,549 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute command Deploy Client Configuration on service KAFKA-1
2019-08-20 12:37:30,566 INFO CommandPusher:com.cloudera.cmf.service.ServiceHandlerRegistry: Executing service command deployClientConfig SvcCmdArgs{targetRoles=[DbRole{id=51, name=CD-KAFKA-ngsMquLo-KAFKA_BROKER-94bb76bb3845d885b0b4b7aef47125e5, hostName=ip-10-0-1-140.ec2.internal}], args=[]}. Service: DbService{id=10, name=CD-KAFKA-ngsMquLo}
2019-08-20 12:37:30,566 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Execute 1 steps in sequence
2019-08-20 12:37:30,566 INFO CommandPusher:com.cloudera.cmf.command.flow.CmdStep: Executing command work: Generate and deploy client configuration.
2019-08-20 12:37:30,583 INFO CommandPusher:com.cloudera.cmf.service.AbstractRoleHandler: Client configs of DbService{id=4, name=CD-ZOOKEEPER-OaAVutjk} wanted but not available.
2019-08-20 12:37:30,720 INFO avro-servlet-hb-processor-0:com.cloudera.server.common.AgentAvroServlet: (45 skipped) AgentAvroServlet: heartbeat processing stats: average=22ms, min=3ms, max=244ms.
2019-08-20 12:37:31,280 INFO avro-servlet-hb-processor-0:com.cloudera.cmf.command.components.StalenessChecker: No staleness check scheduled, scheduling one in 30 seconds
2019-08-20 12:37:32,516 INFO metric-schema-updater:com.cloudera.cmon.MetricSchema: Switching metrics schema from 3yUo1DPTXB4GQELmbFik8-Hfrds to JD_R1wFZTCnbR7qAoUTAM6uoW9k
2019-08-20 12:37:38,048 INFO 257345501@scm-web-24:com.cloudera.server.web.cmf.AuthenticationSuccessEventListener: Authentication success for user: 'admin' from 10.0.1.180 2. from "journalctl -xe" Out of memory: Kill process 14476 (java) score 299 or sacrifice child
Killed process 14476 (java) total-vm:5003380kB, anon-rss:2305344kB, file-rss:0kB, shmem-rss:0kB
... View more
08-20-2019
03:33 AM
Hi All, please help to find out why after changing Java from 1.7 to 1.8.0_181 we can't deploy the cluster and get an error during deploy client configuration. ( There was an error when communicating with the server) Status of service 'cloudera-scm-server' changes from 'active (running)' to 'active (exited)' during the first Running and Deploy client configuration. Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (running) since Tue 2019-08-20 09:30:55 UTC; 16min ago
Docs: man:systemd-sysv-generator(8)
CGroup: /system.slice/cloudera-scm-server.service
├─14471 su cloudera-scm -s /bin/bash -c nohup /usr/sbin/cmf-server
└─14473 /usr/java/jdk1.8.0_181-cloudera/bin/java -cp .:/usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:lib/* -server -Dlog4j.co.. ● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Tue 2019-08-20 09:30:55 UTC; 18min ago All configuration the same, only Java version was changed. We use CDH 5.15.2, Altus Director 2.7, and prebaked image with Java 1.8.0_181 on Centos 7.
... View more
Labels:
- Labels:
-
Cloudera Manager
