From 389 Directory Server

389 uses the Mozilla LDAP C SDK. The OpenLDAP API is similar, but there are a number of important differences. This page lists the differences and the plan for resolving them in order to use the OpenLDAP API with 389 directory server, admin server, adminutil, etc.

Contents 1 LDAP protocol 1.1 LDAP API 1.1.1 Renamed defines 1.1.2 Missing defines 1.1.3 Missing functions 1.1.4 Sort API 1.1.5 URL parsing 1.1.6 ldap_sasl_interactive_bind_ext_s() 1.1.7 ldap_get_lderrno() 1.1.8 ldap_create_proxyauth_control() 1.1.9 ldap_str2charray() 1.1.10 ldap_create_filter() 1.1.11 ldap_init() 1.1.12 ldap_simple_bind_s() 1.1.13 ldap_X deprecated in favor of ldap_X_ext 2 LDAP BER 2.1 ber_len_t 2.2 ber_get_next_buffer_ext() 2.3 LBER_OVERFLOW 3 LDIF 4 UTF-8 5 Crypto 5.1 TLS/SSL 5.2 Password hashing

LDAP protocol

This includes APIs that deal with LDAP protocol requests and responses, such as ldap_search_*, ldap_result, etc. These are mostly the same. There are a few differences such as ldap_initialize, ldap_url_parse.

LDAP API

OpenLDAP supports the BER functions needed to construct and parse the control and extop values. All of the protocol API functions support request and response controls, and extended operation requests and responses. There are a few #defines which are different.

Renamed defines

• LDAP_CHANGETYPE_ADD - LDAP_CONTROL_PERSSIT_ENTRY_CHANGE_ADD
• LDAP_CHANGETYPE_DELETE - LDAP_CONTROL_PERSSIT_ENTRY_CHANGE_DELETE
• LDAP_CHANGETYPE_MODIFY - LDAP_CONTROL_PERSSIT_ENTRY_CHANGE_MODIFY
• LDAP_CHANGETYPE_MODDN - LDAP_CONTROL_PERSSIT_ENTRY_CHANGE_RENAME
• LDAP_CONTROL_PERSISTENTSEARCH - LDAP_CONTROL_PERSIST_REQUEST
• LDAP_X_CONTROL_PWPOLICY_REQUEST - LDAP_CONTROL_PASSWORDPOLICYREQUEST
• LDAP_X_CONTROL_PWPOLICY_RESPONSE - LDAP_CONTROL_PASSWORDPOLICYRESPONSE
• LDAP_FILTER_EXTENDED - LDAP_FILTER_EXT
• LDAP_ALL_USER_ATTRS - LDAP_ALL_USER_ATTRIBUTES

Missing defines

OpenLDAP does not define these:

• LBER_END_OF_SEQORSET
• LDAP_CHANGETYPE_ANY
• LDAP_CONTROL_PWEXPIRED
• LDAP_CONTROL_PWEXPIRING
• LDAP_URL_ERR_NODN - OpenLDAP returns LDAP_URL_ERR_MEM instead if there is no DN in the LDAP URL, but it also uses this value for other, different error cases

Missing functions

OpenLDAP has deprecated these:

• ldap_value_free() - create slapi_ldap_value_free() wrapper
• ldap_count_values() - create slapi_ldap_count_values() wrapper

Sort API

The server side sort API is slightly different. MozLDAP declares this structure:

typedef struct LDAPsortkey {    /* structure for a sort-key */
       char *  sk_attrtype;
       char *  sk_matchruleoid;
       int     sk_reverseorder;
} LDAPsortkey;

But OpenLDAP uses this instead:

typedef struct ldapsortkey {
       char *  attributeType;
       char *  orderingRule;
       int     reverseOrder;
} LDAPSortKey;

We can just typedef LDAPSortkey LDAPsortkey if HAVE_OPENLDAP

URL parsing

The LDAPURLDesc is slightly different. OpenLDAP does not have a lud_options field or a LDAP_URL_OPT_SECURE flag. Instead, OpenLDAP has a lud_scheme field which will have a value of "ldap", "ldaps", or "ldapi".

OpenLDAP does not return LDAP_URL_ERR_NODN if the DN is missing, it will return LDAP_URL_ERR_MEM. Unfortunately, LDAP_URL_ERR_MEM is also returned for other errors. It is up to the caller to determine if there was a problem with the DN by looking at the returned structure.

The slapi function

int slapi_ldap_url_parse(const char *url, LDAPURLDesc **ludpp, int require_dn, int *secure)

is used to parse URLs. If require_dn is set, the url must have a DN or the parse function will return an error. If the secure flag is passed, the function will return secure == TRUE if the url is for a secure protocol (i.e. begins with ldaps:).

ldap_sasl_interactive_bind_ext_s()

OpenLDAP does not have ldap_sasl_interactive_bind_ext_s(), only ldap_sasl_interactive_bind_s(). The only difference is that ldap_sasl_interactive_bind_ext_s() has a LDAPControl ***rctrl parameter which is used to return the response controls from the server. The response controls are useful because without them there is no way to get information about password policy. We could just copy ldap_sasl_interactive_bind_s() from OpenLDAP into slapi and merge in the _ext changes to make it ldap_sasl_interactive_bind_ext_s(). We could attempt to get ldap_sasl_interactive_bind_ext_s() into OpenLDAP. I suspect some resistance due to the response controls parameter, which is sort of a hack, in that it returns the response controls from all of the intermediate ldap_sasl_bind responses.

ldap_get_lderrno()

OpenLDAP does not have this function. The way to get the LDAP error code, matched dn, and error message is to use ldap_parse_result().

ldap_create_proxyauth_control()

OpenLDAP does not have this function. We can copy it into slapi, and submit the function for inclusion in OpenLDAP.

ldap_str2charray()

Use str2charray() in charray.c - export this and str2charray_ext() to slapi as slapi_X.

ldap_create_filter()

referint.c - use PR_smprintf or similar - need to normalize origDN first?

ldap_init()

This is deprecated - have to use ldap_initialize() (converting host:port to an LDAP URL string) or ldap_create() (then set the host:port afterwards).

ldap_simple_bind_s()

This is deprecated - use ldap_sasl_bind_s() with LDAP_SASL_SIMPLE as the mech instead.

ldap_X deprecated in favor of ldap_X_ext

Most of the ldap_X[_s] functions have been deprecated e.g. ldap_unbind(), ldap_add_s(), etc. The _ext versions should be used instead e.g. ldap_unbind -> ldap_unbind_ext.

LDAP BER

OpenLDAP provides BER codecs such as ber_printf, ber_scanf, and many others. In general, OpenLDAP provides a superset of the functionality of MozLDAP, so there should be few difficulties here.

• ber_special_alloc, ber_special_free - Only used in the operation code - move into operation.c
• MozLDAP uses LBER_SOCKET - OpenLDAP uses LBER_SOCKET_T
  • Just use an #ifndef to define LBER_SOCKET
• BER I/O functions - MozLDAP uses struct lextiof_socket_private to pass in the I/O functions that the BER layer should use - OpenLDAP uses struct sockbuf_io
  • code uses #ifdef USE_OPENLDAP - uses sockbuf_io functions for openldap, and the lber lextiof I/O functions for MozLDAP

ber_len_t

MozLDAP defines this as an int, but OpenLDAP defines this as a long. We will have to check all places where this is used to make sure we do not have any 64-bit issues.

ber_get_next_buffer_ext()

OpenLDAP does not provide this function - use ber_get_next() instead. The openldap_read_function will actually "read" from the buffer filled in by connection_read - the openldap_write_function will write to the PRFD. This is why the private data used by the sockbuf IO functions use the Connection* instead of just the PRFD.

LBER_OVERFLOW

If the max incoming BER length is greater than the max, MozLDAP returns LBER_OVERFLOW. OpenLDAP returns LBER_DEFAULT and sets sock_error(ERANGE). Unfortunately, ERANGE is used for other cases as well (e.g. tag specified but length == 0). So it will be a little harder to do max ber detection portably.

LDIF

MozLDAP provides a public API for parsing LDIF files. OpenLDAP has almost the same functions (ldif.c), but they are private to the library (liblutil). A patch has been submitted to expose the LDIF functionality to the public (http://www.openldap.org/its/index.cgi/Incoming?id=6194). There are still some differences. The biggest one is that OpenLDAP has no way to generate LDIF files that are not wrapped. OpenLDAP assumes that since the LDIF is defined as a wrapped format, everyone who uses LDIF must wrap/unwrap. There are a number of places where MozLDAP (ldapsearch -T) or 389 (db2ldif -U) generate unwrapped LDIF. ldif_sput() has a type parameter that takes a bitfield - we could add a LDIF_PUT_NOWRAP option which could be OR'd with the type parameter, and change ldif_sput() to allow unwrapped LDIF output. The initial code in SLAPI will implement the nowrap option as a wrapper around the OpenLDAP function, and will unwrap the LDIF output by ldif_sput(). Since the string length will decrease with unwrapping, the function can just do the unwrap in place. slapi_ldif_put_type_and_value_with_options() is the new wrapper function.

These are different

• LDIF_MAX_LINE_WIDTH - LDIF_LINE_WIDTH
• LDIF_OPT_VALUE_IS_URL - LDIF_PUT_URL

OpenLDAP does not define these:

• LDIF_OPT_NOWRAP - implemented in SLAPI as an option around the OpenLDAP function - will "unwrap" the buffer formatted by OpenLDAP - other apps will need to do something similar
• LDIF_OPT_MINIMAL_ENCODING - not implemented in the OpenLDAP version, so no minimal encoding there - I don't think this will be a problem
• ldif_put_type_and_value - ldif_sput()
• ldif_put_type_and_value_nowrap - ldif_sput()
• ldif_put_type_and_value_with_options - ldif_sput()
• ldif_type_and_value - ldif_sput()
• ldif_type_and_value_nowrap - ldif_sput()
• ldif_type_and_value_with_options - ldif_sput()
• ldif_base64_decode - this is done automatically
• ldif_base64_encode - looks like with OpenLDAP you first register (ldif_must_b64_encode_register()) the attributes that must be base64 encoded, then the "put" routines will automatically base64 encode them, then you have to call ldif_must_b64_encode_release() to "unregister"
  • userPassword is always encoded
• ldif_base64_encode_nowrap
• ldif_get_entry - ldif_read_record()

UTF-8

MozLDAP provides a public API for doing string manipulation of UTF-8 encoded strings (ldap_utf8len(), ldap_utf8strtok_r(), ldap_utf8isalnum(), etc.) OpenLDAP has most of these same functions, but the names are slightly different (e.g. ldap_utf8_next()), and the interface is private to the library (utf-8.c).

For the directory server, we have two options

• copy the UTF-8 API into SLAPI
• export the OpenLDAP utf-8.c and ldap_pvt_uc.h UTF-8 APIs
  • Note that many of the names are different e.g. ldap_utf8_next() instead of ldap_utf8next()

The fastest would be the first option - it would be trivial to make these available via SLAPI. I don't know if there is any motivation in the OpenLDAP community to expose these APIs. A new file - utf8.c - has been added to SLAPI. This is only compiled when using OpenLDAP.

Outside of the server, the only places these are used are adminutil and dsgw. Since both of these also use ICU, and ICU has UTF-8 string functions, one option would be to just use ICU:

• ldap_utf8len() - not used
• ldap_utf8next() - U8_NEXT
• ldap_utf8prev() - U8_PREV
• ldap_utf8copy() - U8_APPEND
• ldap_utf8characters() - not used
• ldap_utf8getcc() - U8_NEXT
• ldap_utf8strtok_r() - none - there is u_strtok_r but that operates on UChar*, not strings of UTF-8 characters
  • this is only used in one place - in dsgw config.c - maybe rewrite that code or just do the UTF-8 to UChar conversion and use u_strtok_r

The ldap_utf8isX functions take a char *, but the u_isX functions take a UChar32. In most cases these functions are used when iterating through a string (e.g. using isspace to trim leading and trailing spaces). These cases will work well when using U8_NEXT and U8_PREV, since those also return the UTF-8 char as a UChar32.

• ldap_utf8isalnum - u_isalnum
• ldap_utf8isalpha - u_isalpha
• ldap_utf8isdigit - u_isdigit with radix 10
• ldap_utf8isxdigit - u_isdigit with radix 16
• ldap_utf8isspace - u_isspace

There are several macros which just wrap the corresponding function. We could just define these in the adminutil and dsgw header files.

• #define LDAP_UTF8LEN(s) ((0x80 & *(unsigned char*)(s)) ? ldap_utf8len (s) : 1)
• #define LDAP_UTF8NEXT(s) ((0x80 & *(unsigned char*)(s)) ? ldap_utf8next(s) : ( s)+1)
• #define LDAP_UTF8INC(s) ((0x80 & *(unsigned char*)(s)) ? s=ldap_utf8next(s) : ++s)
• #define LDAP_UTF8PREV(s) ldap_utf8prev(s)
• #define LDAP_UTF8DEC(s) (s=ldap_utf8prev(s))
• #define LDAP_UTF8COPY(d,s) ((0x80 & *(unsigned char*)(s)) ? ldap_utf8copy(d,s) : ((*(d) = *(s)), 1))
• #define LDAP_UTF8GETCC(s) ((0x80 & *(unsigned char*)(s)) ? ldap_utf8getcc (&s) : *s++)
• #define LDAP_UTF8GETC(s) ((0x80 & *(unsigned char*)(s)) ? ldap_utf8getcc ((const char**)&s) : *s++)

Crypto

TLS/SSL

OpenLDAP CVS HEAD (2.4.17) has support for MozNSS crypto. This works for apps that use the library as well as standalone apps like ldapsearch, etc. and OpenLDAP in server mode. There are some caveats:

• The TLS parameters like cacertdir, cacertfile, certfile, and keyfile have been overloaded to provide parameters for NSS:
  • cacertdir or the directory of cacertfile is used for the NSS key/cert db directory
  • certfile is used for the name of the cert to use (for server or for client auth)
  • keyfile is used for the key/cert db pin text (however, openldap only supports unlocked keys)
• No PEM support yet - still need to integrate support for the PEM PKCS11 library
• No multi-init support yet - NSS upstream is working on this, so for now OpenLDAP is not really suitable for use in apps like nss_ldap

Password hashing