It probably makes more sense to store member information in our LDAP directory rather than in some crappy SQL ORM thing (see: memberdb).

To do this, we need a structured schema to allow us to store the information we want. The current LDAP schema already uses the inetOrgPerson and posixAccount classes, and these provide a useful base (name, email address, account name). We can use these classes to store some of the data we are interested in (such as homePostalAddress, birthday, and rfc822Mailbox), but some of the other data requires us to define our own classes.

LDAP already contains some custom UCC classes for Dispense - MIFARE attributes are stored in a uccDispenseAccount objectClass (not shown below, see /services/ldap/ucc.schema)

When designing a schema, it is important to consider:
 * attribute type
 * search indexes
 * security/ACLs

Only the attribute type is set in the schema; the others are defined in slapd.conf.

= Schema File =
{{{
# UCC's LDAP OID is 1.3.6.1.4.8324.3.2.1.1
objectIdentifier uccLDAP 1.3.6.1.4.8324.3.2.1.1

# LDAP attributes are in 8324.3.2.1.1.1
objectIdentifier uccAttributeType uccLDAP:1
objectIdentifier uccMemberAttribute uccAttributeType:2

# LDAP objectTypes are in 8324.3.2.1.1.2
objectIdentifier uccObjectType uccLDAP:2

####
# UCC member attributes

# UCC TLA
# SYNTAX is type directoryString
# maximum length of 6
attributetype (uccMemberAttribute:1 NAME 'uccMemberTLA'
    DESC 'UCC member three-letter acronym'
    EQUALITY caseIgnoreMatch
    SUBSTR caseIgnoreSubstringsMatch
    ORDERING caseIgnoreOrderingMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{6}
    )
# should probably not be unique, collisions not impossible
# multiple values allowed? not sure
# ACL: writeable by committee, readable by all

# Date account created
# SYNTAX is type generalizedTime
# probably optional because we have a bunch of accounts that don't have this information
attributetype (uccMemberAttribute:2 NAME 'uccAccountCreated'
    DESC 'Date of UCC account creation'
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.24
    EQUALITY generalizedTimeMatch
    ORDERING generalizedTimeOrderingMatch
    SINGLE-VALUE
    )
# ACL: writeable by wheel, readable to user and committee

# Date account last renewed
# SYNTAX is type generalizedTime
# probably mandatory, can be set to a generic value
attributetype (uccMemberAttribute:3 NAME 'uccAccountRenewed'
    DESC 'Date UCC account last renewed'
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.24
    EQUALITY generalizedTimeMatch
    ORDERING generalizedTimeOrderingMatch
    SINGLE-VALUE
    )
# ACL: writeable by wheel and committee, readable to user

# Life membership status
# SYNTAX is type boolean
# probably mandatory, defaults to False
attributetype (uccMemberAttribute:4 NAME 'uccLifeMember'
    DESC 'UCC life membership status'
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.7
    EQUALITY booleanMatch
    SINGLE-VALUE
    )
# ACL: writeable by committee, readable to all? or just wheel/committee/user?
# indexes: exact match

# Student number / UWA Person ID
# SYNTAX is type numericString
attributetype (uccMemberAttribute:5 NAME 'uccUWAPersonID'
    DESC 'UWA Person ID (student/staff number)'
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.36{8}
    EQUALITY numericStringMatch
    ORDERING numericStringOrderingMatch
    SUBSTR numericStringSubstringsMatch
    )
# ACL: writeable by user, perhaps with some restrictions, readable to committee
# probably enforce unique values
# probably allow multiple values e.g. staff number and student number
# indexes: exact match

# Guild member
attributetype (uccMemberAttribute:6 NAME 'uccGuildMember'
    DESC 'UWA Guild of Undergraduates Financial Member'
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.7
    EQUALITY booleanMatch
    SINGLE-VALUE
    )
# ACL: writeable by user, perhaps with some restrictions, readable to committee
# indexes: exact match

# UCC member object
# dob is from labextension.schema
objectclass ( uccObjectType:2 NAME 'uccMember'
    DESC 'UCC member'
    STRUCTURAL
    SUP inetOrgPerson
    MUST (uccAccountRenewed $ uccLifeMember $ uid )
    MAY (uccMemberTLA $ uccAccountCreated $ uccUWAPersonID $ uccGuildMember $ dob $ homePostalAddress $ mail $ telephoneNumber)
    )
}}}

== Comments ==
Put your comments here. [DAA]

uccUWAPersonID should probably be optional, seeing as we have a few members that don't have either a student number or a staff number. This does however present the problem of being able to confirm the identity of new members in this position (assuming it's an online form). [BOB]
 * Yeah, uccUWAPersonID is a MAY rather than a MUST attribute (see the objectClass). I had not intended that this be a way of signing up new members at this stage, not that we do at all a good job of verifying identity anyway. [DAA]

need to clean up the test accounts and things that are in /home/wheel & /home/ucc before using the scripts below [DAA]

not all data from memberdb imports cleanly - the error log should be reviewed and any issues corrected [DAA]

ACLs: there are some things that we don't want people to be able to edit (like their guild status) directly. maybe.

= Migration =
 * Dump the database: `slapcat > slapcat.date`
 * Update `/etc/ldap/schema/ucc.schema` to the new version: `cd /services/ldap && make`
 * Update the dump to the new schema using the following Python script:

{{{#!python
import ldif, sys, csv

class UCCUpgradeLDIF(ldif.LDIFParser):
    def __init__(self, input, output):
        ldif.LDIFParser.__init__(self, input)
        self.writer = ldif.LDIFWriter(output)
    
    def handle(self, dn, entry):
        if 'inetOrgPerson' in entry['objectClass'] and 'uid' in entry and 'ou=Contacts' not in dn:
            uid = entry['uid'][0]
            homedir = entry['homeDirectory'][0]
            
            if '/home/wheel' in homedir or '/home/ucc' in homedir:
                # probably a real person and a UCC member
                entry['objectClass'].remove('inetOrgPerson')
                entry['objectClass'].append('uccMember')
                entry['structuralObjectClass'][0] = 'uccMember'
                entry['uccLifeMember'] = ['FALSE']
                entry['uccAccountRenewed'] = ['197001010000Z']
            
        self.writer.unparse(dn, entry)

parser = UCCUpgradeLDIF(sys.stdin, sys.stdout)
parser.parse()
}}}

 * Stop the master slapd instance on Mussel: `/etc/init.d/slapd stop`
 * Nuke the database: `mv /var/lib/ldap /var/lib/ldap-preschema && mkdir /var/lib/ldap && chown openldap:openldap /var/lib/ldap`
 * Reload the database: `slapadd < slapcat.edited.date && chown openldap:openldap /var/lib/ldap/*`
 * Stop the slave slapd instance on Martello
 * Start the master slapd instance on Mussel: `/etc/init.d/slapd start`
 * Update the schema on Martello
 * Nuke the database on Martello
 * Start the slave slapd instance on Martello (this will pull down the DB and rebuild the local copy as required)

Now we can start adding attributes.

 * Add account creation dates - run this script as `ssh mussel cat /usr/local/newusers.list | uccCreateTime.py | ldapmodify -c -S errorlog -D cn=admin,dc=ucc,dc=gu,dc=uwa,dc=edu,dc=au -x -y /etc/pam_ldap.secret`

{{{#!python
import sys
from datetime import datetime

nl = sys.stdin
out = sys.stdout

for line in nl.readlines():
    if 'failed at' in line:
        continue
    
    uid = line.split(' ')[0]
    datestring = line.split('added on')[1].strip()
    
    createdate = datetime.strptime(datestring, '%a %b %d %H:%M:%S %Z %Y')
    # bodgy hack - assume all times are GMT+0800, as the timezone doesn't get detected properly
    formatted_date = createdate.strftime('%Y%m%d%H%M%S+0800')
    
    out.write("""dn: uid=%s,ou=People,dc=ucc,dc=gu,dc=uwa,dc=edu,dc=au
changetype: modify
add: uccAccountCreated
uccAccountCreated: %s
-

""" % (uid, formatted_date))
}}}

 * Import data from MemberDB - run this script as `uccMemberDBLDIF.py | ldapmodify -c -S errorlog -D cn=admin,dc=ucc,dc=gu,dc=uwa,dc=edu,dc=au -x -y /etc/pam_ldap.secret`

{{{#!python
import sys, os

out = sys.stdout

sys.path += ['/home/wheel/zanchey']
os.environ['DJANGO_SETTINGS_MODULE'] = 'uccmemberdb.settings'

from uccmemberdb.memberdb.models import Member

members = Member.objects.exclude(username='')

for user in members:
    ldif = ''
    ldif += 'dn: uid=%s,ou=People,dc=ucc,dc=gu,dc=uwa,dc=edu,dc=au\n' % user.username
    ldif += 'changetype: modify\n'
    
    if user.address:
        ldif += 'add: homePostalAddress\n'
        ldif += 'homePostalAddress: %s\n' % (user.address.strip().replace('\r\n', '$'))
        ldif += '-\n'
    
    if user.date_of_birth:
        ldif += 'add: dob\n'
        ldif += 'dob: %s\n' % user.date_of_birth.strftime('%Y-%m-%d')
        ldif += '-\n'
    
    if user.email_address:
        ldif += 'add: mail\n'
        ldif += 'mail: %s\n' % user.email_address
        ldif += '-\n'
    
    ldif += 'add: uccGuildMember\n'
    if user.guild_member:
        ldif += 'uccGuildMember: TRUE\n'
    else:    
        ldif += 'uccGuildMember: FALSE\n'
    ldif += '-\n'
    
    if user.phone_number:
        ldif += 'add: telephoneNumber\n'
        ldif += 'telephoneNumber: %s\n' % user.phone_number
        ldif += '-\n'
    
    if user.student_no:
        ldif += 'add: uccUWAPersonID\n'
        ldif += 'uccUWAPersonID: %s\n' % user.student_no
        ldif += '-\n'
        
    ldif += 'replace: uccAccountRenewed\n'
    ldif += 'uccAccountRenewed: %s\n' % (user.signed_up.strftime('%Y%m%d080000+0800')) 
    ldif += '-\n'
    
    ldif += '\n'
    
    out.write(ldif)
}}}