Matching Rule API


Introduction

We need to make the value comparison and index key generation functions matching rule aware. Most places in the server just assume the attribute should just use whatever is provided by the syntax via the syntax plugin, without taking into consideration the EQUALITY, ORDERING, and SUBSTR matching rules defined for the attribute in the schema. The current matching rule plugin API is really only useful for extensible match filters. We need to replace the current server API with one that is matching rule aware.

Functions that deal directly or indirectly with syntax plugins:

slapi_attr_type2plugin(const char *type, void **pi)

called from str2entry_dupcheck

Used to get the syntax plugin for valuetree_add_valuearray() and valuetree_add_value()

Used to get the compare function (via plugin_call_syntax_get_compare_fn) for duplicate comparison

The syntax plugin is passed to valuetree_add_valuearray() and valuetree_add_value() - valuetree_add_value() just calls valuetree_add_valuearray() - valuetree_add_valuearray() uses the plugin to call slapi_call_syntax_values2keys_sv() to get the EQUALITY keys for duplicate checking - so we could just pass in the equality matching rule syntax plugin instead, or better yet, find the right plugin to call based on the given attribute name (first parameter to valuetree_add_valuearray() is const char *type), or if it is too expensive to look up the right plugin to use every time from the attribute type name, have a new function that takes a Slapi_Attr * instead, and store the plugin in the Slapi_Attr.

The compare function is used only for equality - replace with plugin_call_syntax_filter_ava()

called from slapi_attr_get_syntax_oid_copy

Used to get the oid of the syntax of the given attribute - this function is primarily used to determine if the given attribute has DN syntax.

called from get_normalized_value in filtercmp.c

Used to call slapi_call_syntax_values2keys_sv() to get the equality keys

called from ava_candidates in filterindex.c

Used to call slapi_call_syntax_assertion2keys_ava_sv() - looks like only used for EQUALITY or APPROX but not sure

called from range_candidates in filterindex.c

Used to call slapi_call_syntax_assertion2keys_ava() - range implies ORDERING here

called from substring_candidates in filterindex.c

Used to call slapi_call_syntax_assertion2keys_sub_sv - SUBSTR

called from vlv_trim_candidates_byvalue

Used to call slapi_call_syntax_values2keys to get the EQUALITY keys, and to call plugin_call_syntax_get_compare_fn() to get the compare function. The compare_fn here must be able to collate - not just used for equality - that implies ORDERING if available, otherwise fallback to syntax compare function

called from attr_index_config

The struct attrinfo has a member ai_plugin which holds the syntax plugin - we will probably need to add separate plugins for the eq, ord, and sub matching rules - also used to get the compare function if needed (user specified an ordering matching rule in the index config, or the attribute schema def has ORDERING)

*called from sort_candidates

Used to call plugin_call_syntax_get_compare_fn to get the compare function for sort ordering for each attribute -

called from vlvIndex_init

The vlvIndex structure has a member called vlv_syntax_plugin which is an array of syntax plugins, one for each attribute type in the spec

called from slapi_dn_syntax_check

Used to get the dn syntax plugin for syntax validation

called from slapi_mods_syntax_check

Used for syntax validation

slapi_call_syntax_values2keys_sv

called from valuetree_add_valuearray

Used to get the equality keys - vpi is passed in as an argument to valuetree_add_valuearray - see above valuetree_add_valuearray() about how we could find the matching rule plugin instead

called from valuetree_find

Used to get the equality keys - vpi is the struct slapi_attr a_plugin field which is just the base syntax plugin - could store the eq matching rule plugin in struct slapi_attr

called from get_normalized_value

Used to get the equality keys - vpi is from the call to slapi_attr_type2plugin - lookup eq mr plugin from ava type - or possibly store correct plugin to use in struct slapi_filter

called from vlv_create_key

Used to get the equality keys - vpi is from vlv_syntax_plugin which is from slapi_attr_type2plugin from vlvIndex_init (see above) - the function does ordering using slapi_berval_cmp, which assumes the keys returned are normalized, but not necessarily ordered (e.g. could use syntax keys if no ordering matching rule was specified) - could change vlvIndex_init to lookup ORDERING plugin, error and/or fallback to syntax plugin

called from index_addordel_values_ext_sv

Used to generate equality, approx, substr index keys - vpi is from the struct attrinfo ai_plugin field - change struct attrinfo to add eq, ord, and sub mr syntax plugins

called from vlv_build_candidate_list_byvalue

Used to generate the vlv key to look for - vpi is from vlv_syntax_plugin[0] from vlvIndex_Init

called from vlv_trim_candidates_byvalue

See above

slapi_call_syntax_assertion2keys_ava_sv( void *vpi, Slapi_Value *val, Slapi_Value ***ivals, int ftype )

called from ava_candidates

Used to generate keys based on type of comparison - vpi is from call to slapi_attr_type2plugin in same function - probably need to extend slapi_attr_type2plugin to return mr plugin based on type needed

called from range_candidates

Used to generate equality keys for ordering - vpi is from call to slapi_attr_type2plugin in same function - should use ordering plugin if available, or equality, or regular syntax plugin fallback

slapi_call_syntax_assertion2keys_sub

called from substring_candidates

Used to get substring keys - vpi is from call to slapi_attr_type2plugin in same function

struct slapdplugin *plugin_syntax_find( const char *nameoroid )

attr_syntax_create

`a.asi_plugin = plugin_syntax_find( attr_syntax );

read_at_ldif

Used in schema verification, to see if specified syntax is supported

void plugin_syntax_enumerate( SyntaxEnumFunc sef, void *arg )

read_schema_dse

Used with schema_syntax_enum_callback to list all officially supported ldapSyntaxes

struct slapdplugin *slapi_get_global_syntax_plugins()

read_config_dse

Lists plugins in nsslapd-plugin attribute

char *plugin_syntax2oid( struct slapdplugin *pi )

slapi_attr_get_syntax_oid_copy

schema_attr_enum_callback

`syntaxoid = plugin_syntax2oid(asip->asi_plugin);

read_at_ldif

lookup parent syntax for attribute superior

attr_index_config

used to call slapi_matchingrule_is_ordering()

int plugin_call_syntax_get_compare_fn(void *vpi, value_compare_fn_type *compare_fn)

str2entry_dupcheck

for duplicate value checking

vlv_trim_candidates_byvalue

attr_index_config

For key ordering - implies ORDERING

valuearray_minus_valuearray

used for qsort for values - implies ORDERING

sort_candidates

Notes

The server code is mostly hardwired to expect that the equality, ordering, substring, and the corresponding index key generation are handled by the syntax plugin for the attribute. The matching rule plugins are only used with extensible filters, not with regular filters.

I think what we need are parallel functions for each function in plugin_syntax.c that takes a Slapi_Attr as an argument instead of the plugin. Some of them already work that way e.g. plugin_call_syntax_filter_ava(), but some do not e.g. slapi_call_syntax_values2keys(). So for the latter, we would need a function slapi_call_syntax_values2keys_attr() or something like that which takes as the first argument a Slapi_Attr *attr instead of void *plugin.

The Slapi_Attr would be extended to have a field not only for the syntax plugin but also for the equality, ordering, and substring plugin. The current behavior would be preserved by having the functions that use these new fields fall back to the syntax plugin if no eq, ord, or sub plugin was specified for the attribute.

struct asyntaxinfo would be extended the same way - fields for asi_eq_plugin, asi_ord_plugin, and asi_sub_plugin would be added. These would be set when the attribute definitions are read in, much the same way as the asi_plugin field is set based on the syntax oid now.

The tricky part would be changing the way we use functions like valuetree_add_valuearray() and valuetree_add_value() that take the plugin as a value. These functions always want to use the EQ matching and key generation. We would need to make sure we either pass in the eq plugin, or use a different API that takes the attribute name and dynamically looks up which plugin to use based on the usage. There may be other places in the code that look up the plugin first, without any knowledge of what usage the plugin will have, so those places will probably need to be changed to just pass in the attribute name/slapi_attr and usage rather than the plugin.

The new eq, ord, and sub plugins will just be plain old syntax plugins. I had considered having them be matching rule plugins, but the matching rule plugin API is really designed for the extensible matching filters and i18n collation. It would be difficult to make it fit in with the way the syntax plugins are used. Using the syntax plugin API, for example, an eq syntax plugin will not provide functions for slapi_call_syntax_assertion2keys_sub(), only for slapi_call_syntax_assertion2keys_ava(). Conversely, a sub syntax plugin will not provide slapi_call_syntax_assertion2keys_ava(), only slapi_call_syntax_assertion2keys_sub(), etc. That way, we don’t need another plugin type, we shouldn’t need to extend the current syntax plugin API, and it should be easy for the existing syntax plugin code to register these new “matching rule” syntax plugins using their OID and name.

Last modified on 7 August 2014