- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Adding hosts to cluster, found different UID and GID. Is this correct?
- Labels:
-
Apache Ambari
-
Apache Hadoop
Created ‎10-09-2016 07:56 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
First, I setup cluster with 7 hosts. Then I added another 2 hosts which are new installed OS. All done by Ambari UI.
Currently, the 9 hosts runs no problem.
But I noticed the UID and GID on later 2 hosts are some different with the original 7 hosts.
such as:
The original 7 hosts:
zookeeper:x:1000:1000::/home/zookeeper:/bin/bash ams:x:1001:1000::/home/ams:/bin/bash ambari-qa:x:1002:1000::/home/ambari-qa:/bin/bash hdfs:x:1003:1000::/home/hdfs:/bin/bash yarn:x:1004:1000::/home/yarn:/bin/bash mapred:x:1005:1000::/home/mapred:/bin/bash hbase:x:1006:1000::/home/hbase:/bin/bash slider:x:1007:990:SLIDER:/var/lib/slider:/bin/bash hive:x:1008:1000::/home/hive:/bin/bash oozie:x:1009:1000::/home/oozie:/bin/bash tez:x:1010:1000::/home/tez:/bin/bash flume:x:1011:1000::/home/flume:/bin/bash kafka:x:1012:1000::/home/kafka:/bin/bash sqoop:x:1013:1000::/home/sqoop:/bin/bash hcat:x:1014:1000::/home/hcat:/bin/bash falcon:x:996:986:Falcon:/var/lib/falcon:/bin/bash zeppelin:x:1015:1000::/home/zeppelin:/bin/bash livy:x:1016:1000::/home/livy:/bin/bash spark:x:1017:1000::/home/spark:/bin/bash
hadoop:x:1000:zookeeper,ams,hdfs,yarn,mapred,hbase,hive,flume,kafka,sqoop,hcat,zeppelin,livy,spark hdfs:x:1001:hdfs zookeeper:x:994: yarn:x:993: mapred:x:992: hbase:x:991: slider:x:990: flume:x:989: hive:x:988: oozie:x:987: falcon:x:986:falcon livy:x:1002: spark:x:1003: zeppelin:x:1004:
But the later 2 hosts:
hive:x:1000:1003::/home/hive:/bin/bash zookeeper:x:1001:1003::/home/zookeeper:/bin/bash oozie:x:1002:1003::/home/oozie:/bin/bash ams:x:1003:1003::/home/ams:/bin/bash tez:x:1004:1003::/home/tez:/bin/bash zeppelin:x:1005:1003::/home/zeppelin:/bin/bash livy:x:1006:1003::/home/livy:/bin/bash spark:x:1007:1003::/home/spark:/bin/bash ambari-qa:x:1008:1003::/home/ambari-qa:/bin/bash flume:x:1009:1003::/home/flume:/bin/bash kafka:x:1010:1003::/home/kafka:/bin/bash hdfs:x:1011:1003::/home/hdfs:/bin/bash sqoop:x:1012:1003::/home/sqoop:/bin/bash yarn:x:1013:1003::/home/yarn:/bin/bash mapred:x:1014:1003::/home/mapred:/bin/bash hbase:x:1015:1003::/home/hbase:/bin/bash hcat:x:1016:1003::/home/hcat:/bin/bash falcon:x:996:987:Falcon:/var/lib/falcon:/bin/bash slider:x:1017:986:SLIDER:/var/lib/slider:/bin/bash
livy:x:1000: spark:x:1001: zeppelin:x:1002: hadoop:x:1003:hive,zookeeper,ams,zeppelin,livy,spark,flume,kafka,hdfs,sqoop,yarn,mapred,hbase,hcat hdfs:x:1004:hdfs zookeeper:x:994: yarn:x:993: mapred:x:992: flume:x:991: hbase:x:990: hive:x:989: oozie:x:988: falcon:x:987:falcon slider:x:986: sqoop:x:985:
Created ‎10-10-2016 11:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
UID and GID get assigned in order of availability, it probably means you added some services before adding new hosts and therefore new hosts have unmatched IDs. Ambari does not guarantee the IDs will match across nodes, one way to do so is to create service IDs and groups beforehand manually rather then defaulting to Ambari.
Created ‎10-10-2016 11:37 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
UID and GID get assigned in order of availability, it probably means you added some services before adding new hosts and therefore new hosts have unmatched IDs. Ambari does not guarantee the IDs will match across nodes, one way to do so is to create service IDs and groups beforehand manually rather then defaulting to Ambari.
Created ‎10-12-2016 11:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Can we just copy the /etc/passwd and /etc/group of original hosts to the hosts those to be adding?
Created ‎10-12-2016 01:07 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I doubt it is that easy. I'd let OS handle that.
