Created on 05-22-2017 11:44 PM - edited 05-23-2017 01:04 AM
env:
cloudera manager 5.11.0
centos6.4
cdh5.4.0
supervisor 3.0
balancer start with one_off_wrapper.sh
/bin/bash /usr/lib64/cmf/service/common/one_off_wrapper.sh /usr/lib64/cmf/service/hdfs/hdfs.sh balancer -threshold 1.0 -policy DataNode hdfs 17593 15.2 2.8 2863588 705044 ? Sl 14:36 0:09 \_ /usr/java/jdk1.8/bin/java -Dproc_balancer -Xmx1000m -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/home/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hadoop/logs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,console -Djava.library.path=/home/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Xms1073741824 -Xmx1073741824 -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.hdfs.server.balancer.Balancer -threshold 1.0 -policy DataNode
When I stop balancer from website, the one_off_wrapper.sh killed, but java process remain.
supervisor conf
[program:706-hdfs-BALANCER] command=/usr/lib64/cmf/service/common/one_off_wrapper.sh "/usr/lib64/cmf/service/hdfs/hdfs.sh" "balancer" "-threshold" "1.0" "-policy" "DataNode" autostart=true directory=/var/run/cloudera-scm-agent/process/706-hdfs-BALANCER stdout_logfile=/var/run/cloudera-scm-agent/process/706-hdfs-BALANCER/logs/stdout.log stdout_logfile_maxbytes=10MB stdout_logfile_backups=10 stderr_logfile=/var/run/cloudera-scm-agent/process/706-hdfs-BALANCER/logs/stderr.log stderr_logfile_maxbytes=10MB stderr_logfile_backups=10 environment=CDH_PARQUET_HOME='/usr/lib/parquet',CDH_MR2_HOME='/usr/lib/hadoop-mapreduce',CMF_PACKAGE_DIR='/usr/lib64/cmf/service',CDH_HADOOP_BIN='/usr/bin/hadoop',MGMT_HOME='/usr/share/cmf',CDH_IMPALA_HOME=' /usr/lib/impala',CDH_YARN_HOME='/usr/lib/hadoop-yarn',CDH_KAFKA_HOME='/usr/lib/kafka',CDH_HDFS_HOME='/usr/lib/hadoop-hdfs',HOME='/var/lib/hadoop-hdfs',PATH='/sbin:/usr/sbin:/bin:/usr/bin',CDH_HUE_PLUGINS_HOM E='/usr/lib/hadoop',PARCEL_DIRNAMES='CDH-5.4.0-1.cdh5.4.0.p0.27',CM_STATUS_CODES='STATUS_NONE HDFS_DFS_DIR_NOT_EMPTY HBASE_TABLE_DISABLED HBASE_TABLE_ENABLED JOBTRACKER_IN_STANDBY_MODE YARN_RM_IN_STANDBY_MOD E',KEYTRUSTEE_KP_HOME='/usr/share/keytrustee-keyprovider',CLOUDERA_ORACLE_CONNECTOR_JAR='/usr/share/java/oracle-connector-java.jar',CONF_DIR='/var/run/cloudera-scm-agent/process/706-hdfs-BALANCER',CDH_SQOOP2 _HOME='/usr/lib/sqoop2',KEYTRUSTEE_SERVER_HOME='/usr/lib/keytrustee-server',JSVC_HOME='/usr/libexec/bigtop-utils',HIVE_DEFAULT_XML='/etc/hive/conf.dist/hive-default.xml',JAVA_HOME='/usr/java/jdk1.8',CLOUDERA _POSTGRESQL_JDBC_JAR='/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar',PARCELS_ROOT='/home/cloudera/parcels',CDH_KMS_HOME='/usr/lib/hadoop-kms',CDH_HBASE_HOME='/usr/lib/hbase',CDH_SQOOP_HOME='/usr/lib/sqoop' ,WEBHCAT_DEFAULT_XML='/etc/hive-webhcat/conf.dist/webhcat-default.xml',CDH_OOZIE_HOME='/usr/lib/oozie',CDH_ZOOKEEPER_HOME='/usr/lib/zookeeper',CDH_HUE_HOME='/usr/lib/hue',CLOUDERA_MYSQL_CONNECTOR_JAR='/usr/s hare/java/mysql-connector-java.jar',CDH_SPARK_HOME='/usr/lib/spark',HADOOP_BALANCER_OPTS='-Xms1073741824 -Xmx1073741824 ',CDH_MR1_HOME='/usr/lib/hadoop-0.20-mapreduce',TOMCAT_HOME='/usr/lib/bigtop-tomcat',CD H_PIG_HOME='/usr/lib/pig',CDH_VERSION='5',HADOOP_ROOT_LOGGER='INFO,console',CDH_CRUNCH_HOME='/usr/lib/crunch',CDH_LLAMA_HOME='/usr/lib/llama/',CDH_HTTPFS_HOME='/usr/lib/hadoop-httpfs',CDH_HADOOP_HOME='/usr/l ib/hadoop',CDH_HIVE_HOME='/usr/lib/hive',ORACLE_HOME='/usr/share/oracle/instantclient',CDH_HCAT_HOME='/usr/lib/hcatalog',CDH_SENTRY_HOME='/usr/lib/sentry',CDH_HBASE_INDEXER_HOME='/usr/lib/hbase-solr',SCM_DEF INES_SCRIPTS='/home/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/meta/cdh_env.sh',CDH_SOLR_HOME='/usr/lib/solr',CDH_FLUME_HOME='/usr/lib/flume-ng' user=hdfs group=hdfs exitcodes=0,2 autorestart=false startsecs=0 killasgroup=true # rlimits
agent log
[23/May/2017 14:15:12 +0000] 4950 MainThread process INFO [706-hdfs-BALANCER] Refreshing process files: None [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO prepare_environment begin: {u'PREPROCESS': u'V1.2', u'HDFSFIX': u'5.4.0-1.cdh5.4.0.p2', u'KAFKA': u'2.1.1-1.2.1.1.p0.18', u'CDH': u'5.4.0-1. cdh5.4.0.p0.27', u'CUSTOM': u'5.4.0-1.cdh5.4.0.p2.3'}, [u'cdh'], [u'cdh-plugin', u'hdfs-plugin'] [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO The following requested parcels are not available: {} [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO Obtained tags [] for parcel PREPROCESS [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO Obtained tags [] for parcel HDFSFIX [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO Obtained tags [] for parcel CUSTOM [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO Obtained tags ['cdh', 'impala', 'sentry', 'solr', 'spark'] for parcel CDH [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO Obtained tags ['kafka'] for parcel KAFKA [23/May/2017 14:15:12 +0000] 4950 MainThread parcel INFO prepare_environment end: {'CDH': '5.4.0-1.cdh5.4.0.p0.27'} [23/May/2017 14:15:12 +0000] 4950 MainThread __init__ INFO Extracted 6 files and 0 dirs to /var/run/cloudera-scm-agent/process/706-hdfs-BALANCER. [23/May/2017 14:15:12 +0000] 4950 MainThread process INFO [706-hdfs-BALANCER] Launching process. one-off True, command hdfs/hdfs.sh, args [u'balancer', u'-threshold', u'1.0', u'-policy', u'DataNode' ] [23/May/2017 14:15:12 +0000] 4950 MainThread agent INFO Triggering supervisord update. [23/May/2017 14:15:12 +0000] 4950 MainThread process INFO Begin audit plugin refresh [23/May/2017 14:15:12 +0000] 4950 MainThread navigator_plugin INFO Scheduling a refresh for Audit Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:15:12 +0000] 4950 MainThread process INFO Begin metadata plugin refresh [23/May/2017 14:15:12 +0000] 4950 MainThread navigator_plugin INFO Scheduling a refresh for Metadata Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:15:12 +0000] 4950 MainThread process INFO Not creating a monitor for 706-hdfs-BALANCER: should_monitor returns false [23/May/2017 14:15:12 +0000] 4950 MainThread process INFO Daemon refresh complete for process 706-hdfs-BALANCER. [23/May/2017 14:15:14 +0000] 4950 Audit-Plugin navigator_plugin INFO Refreshing Audit Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:15:14 +0000] 4950 Audit-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for hdfs-BALANCER (log dir: None) [23/May/2017 14:15:14 +0000] 4950 Metadata-Plugin navigator_plugin INFO Refreshing Metadata Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:15:14 +0000] 4950 Metadata-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for hdfs-BALANCER (log dir: None) [23/May/2017 14:20:58 +0000] 4950 MainThread process INFO [706-hdfs-BALANCER] Updating process: False {u'running': (True, False), u'run_generation': (1, 2)} [23/May/2017 14:20:58 +0000] 4950 MainThread process INFO Deactivating process 706-hdfs-BALANCER [23/May/2017 14:20:59 +0000] 4950 MainThread agent INFO Triggering supervisord update. [23/May/2017 14:20:59 +0000] 4950 Audit-Plugin navigator_plugin INFO stopping Audit Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:20:59 +0000] 4950 Audit-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for hdfs-BALANCER (log dir: None) [23/May/2017 14:20:59 +0000] 4950 Metadata-Plugin navigator_plugin INFO stopping Metadata Plugin for hdfs-BALANCER with pipelines [] [23/May/2017 14:20:59 +0000] 4950 Metadata-Plugin navigator_plugin_pipeline INFO Stopping Navigator Plugin Pipeline '' for hdfs-BALANCER (log dir: None) [23/May/2017 14:24:59 +0000] 4950 MainThread heartbeat_tracker INFO HB stats (seconds): num:43 LIFE_MIN:0.07 min:0.03 mean:0.07 max:0.21 LIFE_MAX:0.15
Created 06-20-2017 11:48 PM
anyone has same problem?