Created 10-09-2016 07:56 AM
First, I setup cluster with 7 hosts. Then I added another 2 hosts which are new installed OS. All done by Ambari UI.
Currently, the 9 hosts runs no problem.
But I noticed the UID and GID on later 2 hosts are some different with the original 7 hosts.
such as:
The original 7 hosts:
zookeeper:x:1000:1000::/home/zookeeper:/bin/bash ams:x:1001:1000::/home/ams:/bin/bash ambari-qa:x:1002:1000::/home/ambari-qa:/bin/bash hdfs:x:1003:1000::/home/hdfs:/bin/bash yarn:x:1004:1000::/home/yarn:/bin/bash mapred:x:1005:1000::/home/mapred:/bin/bash hbase:x:1006:1000::/home/hbase:/bin/bash slider:x:1007:990:SLIDER:/var/lib/slider:/bin/bash hive:x:1008:1000::/home/hive:/bin/bash oozie:x:1009:1000::/home/oozie:/bin/bash tez:x:1010:1000::/home/tez:/bin/bash flume:x:1011:1000::/home/flume:/bin/bash kafka:x:1012:1000::/home/kafka:/bin/bash sqoop:x:1013:1000::/home/sqoop:/bin/bash hcat:x:1014:1000::/home/hcat:/bin/bash falcon:x:996:986:Falcon:/var/lib/falcon:/bin/bash zeppelin:x:1015:1000::/home/zeppelin:/bin/bash livy:x:1016:1000::/home/livy:/bin/bash spark:x:1017:1000::/home/spark:/bin/bash
hadoop:x:1000:zookeeper,ams,hdfs,yarn,mapred,hbase,hive,flume,kafka,sqoop,hcat,zeppelin,livy,spark hdfs:x:1001:hdfs zookeeper:x:994: yarn:x:993: mapred:x:992: hbase:x:991: slider:x:990: flume:x:989: hive:x:988: oozie:x:987: falcon:x:986:falcon livy:x:1002: spark:x:1003: zeppelin:x:1004:
But the later 2 hosts:
hive:x:1000:1003::/home/hive:/bin/bash zookeeper:x:1001:1003::/home/zookeeper:/bin/bash oozie:x:1002:1003::/home/oozie:/bin/bash ams:x:1003:1003::/home/ams:/bin/bash tez:x:1004:1003::/home/tez:/bin/bash zeppelin:x:1005:1003::/home/zeppelin:/bin/bash livy:x:1006:1003::/home/livy:/bin/bash spark:x:1007:1003::/home/spark:/bin/bash ambari-qa:x:1008:1003::/home/ambari-qa:/bin/bash flume:x:1009:1003::/home/flume:/bin/bash kafka:x:1010:1003::/home/kafka:/bin/bash hdfs:x:1011:1003::/home/hdfs:/bin/bash sqoop:x:1012:1003::/home/sqoop:/bin/bash yarn:x:1013:1003::/home/yarn:/bin/bash mapred:x:1014:1003::/home/mapred:/bin/bash hbase:x:1015:1003::/home/hbase:/bin/bash hcat:x:1016:1003::/home/hcat:/bin/bash falcon:x:996:987:Falcon:/var/lib/falcon:/bin/bash slider:x:1017:986:SLIDER:/var/lib/slider:/bin/bash
livy:x:1000: spark:x:1001: zeppelin:x:1002: hadoop:x:1003:hive,zookeeper,ams,zeppelin,livy,spark,flume,kafka,hdfs,sqoop,yarn,mapred,hbase,hcat hdfs:x:1004:hdfs zookeeper:x:994: yarn:x:993: mapred:x:992: flume:x:991: hbase:x:990: hive:x:989: oozie:x:988: falcon:x:987:falcon slider:x:986: sqoop:x:985:
Created 10-10-2016 11:37 PM
UID and GID get assigned in order of availability, it probably means you added some services before adding new hosts and therefore new hosts have unmatched IDs. Ambari does not guarantee the IDs will match across nodes, one way to do so is to create service IDs and groups beforehand manually rather then defaulting to Ambari.
Created 10-10-2016 11:37 PM
UID and GID get assigned in order of availability, it probably means you added some services before adding new hosts and therefore new hosts have unmatched IDs. Ambari does not guarantee the IDs will match across nodes, one way to do so is to create service IDs and groups beforehand manually rather then defaulting to Ambari.
Created 10-12-2016 11:34 AM
Can we just copy the /etc/passwd and /etc/group of original hosts to the hosts those to be adding?
Created 10-12-2016 01:07 PM
I doubt it is that easy. I'd let OS handle that.