question Re: How do I automate the Ambari LDAP sync? in Support Questions

How do I automate the Ambari LDAP sync?

SQLShaw — Fri, 06 Nov 2015 02:25:31 GMT

I want to automate through cron or other method "ambari-server sync-ldap --existing" but it prompts for an Ambari username and password. Any ideas on how I can automate an Ambari LDAP sync?

Re: How do I automate the Ambari LDAP sync?

abajwa — Fri, 06 Nov 2015 02:32:40 GMT

@Sean Roberts had scripted this out. Steps here: https://github.com/seanorama/masterclass/tree/master/security#lab-configure-ambari-for-ldap

Here is the script itself

https://github.com/seanorama/ambari-bootstrap/blob/master/extras/ambari-ldap-ad.sh

Re: How do I automate the Ambari LDAP sync?

SQLShaw — Fri, 06 Nov 2015 02:40:06 GMT

That's sets up LDAP but I need to automate the sync process. I would like Ambari to execute the sync-ldap --existing command once a day.

Re: How do I automate the Ambari LDAP sync?

pcodding — Fri, 06 Nov 2015 02:46:18 GMT

You can create an expect script for it.

Re: How do I automate the Ambari LDAP sync?

yusaku — Fri, 06 Nov 2015 03:20:30 GMT

Try:

curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"principal_type": "users", "sync_type": "existing"}, {"principal_type": "groups", "sync_type": "existing"}]}}]' http://localhost:8080/api/v1/ldap_sync_events

You will get a response like:

{
  "resources" : [
    {
      "href" : "http://localhost:8080/api/v1/ldap_sync_events/13",
      "Event" : {
        "id" : 13
      }
    }
  ]
}

You can GET on this href to get status of the sync:

curl -uadmin:admin http://localhost:8080/api/v1/ldap_sync_events/13


{
  "href" : "http://localhost:8080/api/v1/ldap_sync_events/13",
  "Event" : {
    "id" : 13,
    "specs" : [
      {
        "sync_type" : "existing",
        "principal_type" : "users"
      },
      {
        "sync_type" : "existing",
        "principal_type" : "groups"
      }
    ],
    "status" : "COMPLETE",
    "status_detail" : "Completed LDAP sync.",
    "summary" : {
      "groups" : {
        "created" : 0,
        "removed" : 0,
        "updated" : 0
      },
      "memberships" : {
        "created" : 0,
        "removed" : 0
      },
      "users" : {
        "created" : 0,
        "removed" : 0,
        "updated" : 0
      }
    },
    "sync_time" : {
      "end" : 1446751142546,
      "start" : 1446751142462
    }
  }
}

Re: How do I automate the Ambari LDAP sync?

pcodding — Fri, 06 Nov 2015 03:57:46 GMT

@yusaku@hortonworks.com - how would we do this when we have a users or groups text file?

Re: How do I automate the Ambari LDAP sync?

yusaku — Fri, 06 Nov 2015 04:08:03 GMT

You can replace sync_type to specific (from existing), and add names attribute with a comma-delimited list of users/groups. Here's an example:

curl -uadmin:admin -H 'X-Requested-By: ambari' -X POST -d '[{"Event": {"specs": [{"principal_type": "users", "sync_type": "specific", "names": "bill,jenny,mike"},{"principal_type":"groups","sync_type":"specific", "names": "group1,group2"}]}}]' http://localhost:8080/api/v1/ldap_sync_events

Re: How do I automate the Ambari LDAP sync?

nsabharwal — Mon, 19 Aug 2019 12:52:29 GMT

@Scott Shaw

This will make life easier..gist link

yum install expect*

#!/usr/bin/expect

spawn ambari-server sync-ldap --existing

expect "Enter Ambari Admin login:"

send "admin\r"

expect "Enter Ambari Admin password:"

send "admin\r"

expect eof

Re: How do I automate the Ambari LDAP sync?

SQLShaw — Fri, 06 Nov 2015 23:00:22 GMT

Here is the .sh script we used at the customers. You'll need to fill in your specific environment information. It runs the curl commands but also includes an LDAP filter.

#!/bin/sh

# Just in case we are run from cron with no path set...

export PATH=/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/bin

AMBARI_ADMIN_USER='admin:xxxxx'

# # Groups we want to look for, in this case any group name that starts with HDP_ # GROUP_FILTER="(&(ObjectClass=Group)(CN=HDP_*))" SEARCH_BASE=DC=MYDOMAIN,DC=com SEARCH_USER=CN=search_user,OU=LDAP,${SEARCH_BASE} SEARCH_PASSWD=ldapUserPassword

LDAP_HOST=ldap.mydomain.com

LDAPGROUPS=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${GROUP_FILTER}" cn | grep ^dn: | cut -d' ' -f2- | sed -e "s/$.*$/(memberOf=\1)/" | tr '\n' ':' | sed -e "s/://g"`

# Filter for users with a "valid" flag set who have a first name, last name and email. SEARCH_FILTER="(&(objectClass=USER)(mail=*mydomain.com)(givenName=*)(sn=*)(!(msexchuserAccountControl:1.2.840.113556.1.4.803:=2))(|$LDAPGROUPS))"

# perform the search on AD and format the results in a way that postfix wants.

#ldapsearch -h <ldap.company.com> -x -D “${SEARCH_USER}” -w “${SEARCH_PASSWD}” "${SEARCH_FILTER}" sAMAccountName | \ grep -v "{" | \ cut -d: -f3 | \ sort -u > ${TEMP_FILE}

USERLIST=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${SEARCH_FILTER}" sAMAccountName |\ grep -i sAMAccountName |\ grep -v ^# |\ sort -u |\ awk '{print $2}' |\ tr '\n' , |\ tr '[A-Z]' '[a-z]' |\ sed -e "s/,$//"`

GROUPLIST=`ldapsearch -h $LDAP_HOST -x -s sub -b ${SEARCH_BASE} -D ${SEARCH_USER} -w ${SEARCH_PASSWD} "${SEARCH_FILTER}" memberOf |\ grep memberOf |\ grep -v ^# |\ grep HDP_ |\ sort -u |\ cut -d: -f2 |\ cut -d= -f2 |\ cut -d, -f1 |\ awk '{print $1}' |\ tr '\n' , |\ sed -e "s/,$//"`

# Sync new users and groups

curl -s -H "X-Requested-By: ambari” -u $AMBARI_ADMIN_USER -d '{"Event": {"specs": [{"principal_type": "users", "sync_type": "specific", "names": "'$USERLIST'"}, {"principal_type": "groups", "sync_type": "specific", "names": "'$GROUPLIST'"}]}}' http://127.0.0.1:8080/api/v1/ldap_sync_events >/dev/null

sleep 30

# Sync existing users and groups

curl -s -H "X-Requested-By: amber” -u $AMBARI_ADMIN_USER -d '{"Event": { "specs": [{"principal_type": "users", "sync_type": "existing"}, {"principal_type": "groups", "sync_type": "existing"}]}}' http://127.0.0.1:8080/api/v1/ldap_sync_events >/dev/null

echo "AmbariLdapSync complete at `date`"

exit 0

Re: How do I automate the Ambari LDAP sync?

nsabharwal — Fri, 06 Nov 2015 23:02:52 GMT

@Scott Shaw This looks great. Thanks for sharing it.

Re: How do I automate the Ambari LDAP sync?

stijnvd — Fri, 16 Dec 2016 19:26:22 GMT

Both ldapsearch and cron will show up in ps with passwords in their command line. That's easy to prevent with both tools: use the -y option for ldapsearch, and --netrc-file for cron.

Re: How do I automate the Ambari LDAP sync?

rburagohain — Fri, 23 Dec 2016 19:41:08 GMT

@Neeraj Sabharwal

Hi Neeraj,

I have used your ambari-ldap sync script but I get the following error when I ran the below command. One thing I noticed is that if the run the script manually as ./ambari_ldap_sync_all.sh then its getting executed.

Also I have shown my ambari-ldap sync script below. So the script is not getting executed from crontab with 'sh' command . Please help.

[root@host1(172.23.34.4)] # sh ambari_ldap_sync_all.sh
ambari_ldap_sync_all.sh: line 3: spawn: command not found
couldn't read file "Enter Ambari Admin login:": no such file or directory
ambari_ldap_sync_all.sh: line 7: send: command not found
couldn't read file "Enter Ambari Admin password:": no such file or directory
ambari_ldap_sync_all.sh: line 11: send: command not found
couldn't read file "eof": no such file or directory

[root@host1(172.23.34.4)] # cat ambari_ldap_sync_all.sh
#!/usr/bin/expect

spawn ambari-server sync-ldap --existing

expect "Enter Ambari Admin login:"

send "admin\r"

expect "Enter Ambari Admin password:"

send "admin\r"

expect eof

[root@host1(172.23.34.4)] # crontab -e
00 15 * * * /ambari_ldap_sync_all.sh

Re: How do I automate the Ambari LDAP sync?

michael_salmon — Wed, 22 Feb 2017 14:45:46 GMT

This is an expect script not a shell script. Your shell does not understand expect commands.

Re: How do I automate the Ambari LDAP sync?

rbailey — Sun, 09 Apr 2017 19:55:38 GMT

Worth knowing that now there is no need for the "expect" statement now with the following attributes that can be added to the sync-ldap request:

--ldap-sync-admin-name=admin --ldap-sync-admin-password=secret

Re: How do I automate the Ambari LDAP sync?

stijnvd — Fri, 21 Apr 2017 17:28:13 GMT

The unexpected benefit of this is that nobody will ever forget the LDAP password again: not will it be included in your favourite shell's history file, but anyone who can log in on that node will also be able to see those options by keeping an eye on ps. Isn't that neat?

Don't do this, kids. Never write passwords on the command line.