Created 11-05-2015 06:25 PM
I want to automate through cron or other method "ambari-server sync-ldap --existing" but it prompts for an Ambari username and password. Any ideas on how I can automate an Ambari LDAP sync?
Created 11-05-2015 07:20 PM
Try:
curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"principal_type": "users", "sync_type": "existing"}, {"principal_type": "groups", "sync_type": "existing"}]}}]' http://localhost:8080/api/v1/ldap_sync_events
You will get a response like:
{ "resources" : [ { "href" : "http://localhost:8080/api/v1/ldap_sync_events/13", "Event" : { "id" : 13 } } ] }
You can GET on this href to get status of the sync:
curl -uadmin:admin http://localhost:8080/api/v1/ldap_sync_events/13 { "href" : "http://localhost:8080/api/v1/ldap_sync_events/13", "Event" : { "id" : 13, "specs" : [ { "sync_type" : "existing", "principal_type" : "users" }, { "sync_type" : "existing", "principal_type" : "groups" } ], "status" : "COMPLETE", "status_detail" : "Completed LDAP sync.", "summary" : { "groups" : { "created" : 0, "removed" : 0, "updated" : 0 }, "memberships" : { "created" : 0, "removed" : 0 }, "users" : { "created" : 0, "removed" : 0, "updated" : 0 } }, "sync_time" : { "end" : 1446751142546, "start" : 1446751142462 } } }
Created 11-05-2015 06:32 PM
@Sean Roberts had scripted this out. Steps here: https://github.com/seanorama/masterclass/tree/master/security#lab-configure-ambari-for-ldap
Here is the script itself
https://github.com/seanorama/ambari-bootstrap/blob/master/extras/ambari-ldap-ad.sh
Created 11-05-2015 06:40 PM
That's sets up LDAP but I need to automate the sync process. I would like Ambari to execute the sync-ldap --existing command once a day.
Created 11-05-2015 06:46 PM
You can create an expect script for it.
Created 11-05-2015 07:20 PM
Try:
curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"principal_type": "users", "sync_type": "existing"}, {"principal_type": "groups", "sync_type": "existing"}]}}]' http://localhost:8080/api/v1/ldap_sync_events
You will get a response like:
{ "resources" : [ { "href" : "http://localhost:8080/api/v1/ldap_sync_events/13", "Event" : { "id" : 13 } } ] }
You can GET on this href to get status of the sync:
curl -uadmin:admin http://localhost:8080/api/v1/ldap_sync_events/13 { "href" : "http://localhost:8080/api/v1/ldap_sync_events/13", "Event" : { "id" : 13, "specs" : [ { "sync_type" : "existing", "principal_type" : "users" }, { "sync_type" : "existing", "principal_type" : "groups" } ], "status" : "COMPLETE", "status_detail" : "Completed LDAP sync.", "summary" : { "groups" : { "created" : 0, "removed" : 0, "updated" : 0 }, "memberships" : { "created" : 0, "removed" : 0 }, "users" : { "created" : 0, "removed" : 0, "updated" : 0 } }, "sync_time" : { "end" : 1446751142546, "start" : 1446751142462 } } }
Created 11-05-2015 07:57 PM
@yusaku@hortonworks.com - how would we do this when we have a users or groups text file?
Created 11-05-2015 08:08 PM
You can replace sync_type to specific (from existing), and add names attribute with a comma-delimited list of users/groups. Here's an example:
curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"principal_type": "users", "sync_type": "specific", "names": "bill,jenny,mike"},{"principal_type":"groups","sync_type":"specific", "names": "group1,group2"}]}}]' http://localhost:8080/api/v1/ldap_sync_events
Created on 11-05-2015 11:58 PM - edited 08-19-2019 05:52 AM
This will make life easier..gist link
yum install expect*
#!/usr/bin/expect
spawn ambari-server sync-ldap --existing
expect "Enter Ambari Admin login:"
send "admin\r"
expect "Enter Ambari Admin password:"
send "admin\r"
expect eof
Created 11-06-2015 03:00 PM
Here is the .sh script we used at the customers. You'll need to fill in your specific environment information. It runs the curl commands but also includes an LDAP filter.
#!/bin/sh
# Just in case we are run from cron with no path set...
export PATH=/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin
AMBARI_ADMIN_USER='admin:xxxxx'
# # Groups we want to look for, in this case any group name that starts with HDP_ # GROUP_FILTER="(&(ObjectClass=Group)(CN=HDP_*))" SEARCH_BASE=DC=MYDOMAIN,DC=com SEARCH_USER=CN=search_user,OU=LDAP,${SEARCH_BASE} SEARCH_PASSWD=ldapUserPassword
LDAP_HOST=ldap.mydomain.com
LDAPGROUPS=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${GROUP_FILTER}" cn | grep ^dn: | cut -d' ' -f2- | sed -e "s/\(.*\)/(memberOf=\1)/" | tr '\n' ':' | sed -e "s/://g"`
# Filter for users with a "valid" flag set who have a first name, last name and email. SEARCH_FILTER="(&(objectClass=USER)(mail=*mydomain.com)(givenName=*)(sn=*)(!(msexchuserAccountControl:1.2.840.113556.1.4.803:=2))(|$LDAPGROUPS))"
# perform the search on AD and format the results in a way that postfix wants.
#ldapsearch -h <ldap.company.com> -x -D “${SEARCH_USER}” -w “${SEARCH_PASSWD}” "${SEARCH_FILTER}" sAMAccountName | \ grep -v "{" | \ cut -d: -f3 | \ sort -u > ${TEMP_FILE}
USERLIST=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${SEARCH_FILTER}" sAMAccountName |\ grep -i sAMAccountName |\ grep -v ^# |\ sort -u |\ awk '{print $2}' |\ tr '\n' , |\ tr '[A-Z]' '[a-z]' |\ sed -e "s/,$//"`
GROUPLIST=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${SEARCH_FILTER}" memberOf |\ grep memberOf |\ grep -v ^# |\ grep HDP_ |\ sort -u |\ cut -d: -f2 |\ cut -d= -f2 |\ cut -d, -f1 |\ awk '{print $1}' |\ tr '\n' , |\ sed -e "s/,$//"`
# Sync new users and groups
curl -s -H "X-Requested-By: ambari” -u $AMBARI_ADMIN_USER -d '{"Event": {"specs": [{"principal_type": "users", "sync_type": "specific", "names": "'$USERLIST'"}, {"principal_type": "groups", "sync_type": "specific", "names": "'$GROUPLIST'"}]}}' http://127.0.0.1:8080/api/v1/ldap_sync_events >/dev/null
sleep 30
# Sync existing users and groups
curl -s -H "X-Requested-By: amber” -u $AMBARI_ADMIN_USER -d '{"Event": { "specs": [{"principal_type": "users", "sync_type": "existing"}, {"principal_type": "groups", "sync_type": "existing"}]}}' http://127.0.0.1:8080/api/v1/ldap_sync_events >/dev/null
echo "AmbariLdapSync complete at `date`"
exit 0
Created 11-06-2015 03:02 PM
@Scott Shaw This looks great. Thanks for sharing it.
Created 12-16-2016 11:26 AM
Both ldapsearch and cron will show up in ps with passwords in their command line. That's easy to prevent with both tools: use the -y option for ldapsearch, and --netrc-file for cron.
Created 12-23-2016 11:41 AM
Hi Neeraj,
I have used your ambari-ldap sync script but I get the following error when I ran the below command. One thing I noticed is that if the run the script manually as ./ambari_ldap_sync_all.sh then its getting executed.
Also I have shown my ambari-ldap sync script below. So the script is not getting executed from crontab with 'sh' command . Please help.
[root@host1(172.23.34.4)] # sh ambari_ldap_sync_all.sh ambari_ldap_sync_all.sh: line 3: spawn: command not found couldn't read file "Enter Ambari Admin login:": no such file or directory ambari_ldap_sync_all.sh: line 7: send: command not found couldn't read file "Enter Ambari Admin password:": no such file or directory ambari_ldap_sync_all.sh: line 11: send: command not found couldn't read file "eof": no such file or directory [root@host1(172.23.34.4)] # cat ambari_ldap_sync_all.sh #!/usr/bin/expect spawn ambari-server sync-ldap --existing expect "Enter Ambari Admin login:" send "admin\r" expect "Enter Ambari Admin password:" send "admin\r" expect eof [root@host1(172.23.34.4)] # crontab -e 00 15 * * * /ambari_ldap_sync_all.sh
Created 02-22-2017 06:45 AM
This is an expect script not a shell script. Your shell does not understand expect commands.
Created 04-09-2017 12:55 PM
Worth knowing that now there is no need for the "expect" statement now with the following attributes that can be added to the sync-ldap request:
--ldap-sync-admin-name=admin --ldap-sync-admin-password=secret
Created 04-21-2017 10:28 AM
The unexpected benefit of this is that nobody will ever forget the LDAP password again: not will it be included in your favourite shell's history file, but anyone who can log in on that node will also be able to see those options by keeping an eye on ps. Isn't that neat?
Don't do this, kids. Never write passwords on the command line.