One issue that has been raised in regards to SPML is search filters. SPML allows searches that optionally specify a starting point (in terms of an SPML container), a subset of data to return, and a search filter. In the DSML Profile, the search filter is naturally a DSML filter.
DSML filters can be arbitrarily complex, just like the LDAP filters they model. For instances a DSML filter could be something like “get everyone with the last name of smith”, or in DSML:
<filter xmlns=’urn:oasis:names:tc:DSML:2:0:core’>
<substrings name=’Name’><final>Smith</final></substrings>
</filter>
Or it could be “get everyone with last name smith not located in Orlando”:
<filter xmlns=’urn:oasis:names:tc:DSML:2:0:core’>
<and>
<substrings name=’Name’><final>Smith</final></substrings>
<not>
<substrings name=’Office’>
<final>Orlando</final>
</substrings>
</not>
</and>
</filter>
Now if your back end data is stored in LDAP, then this is pretty easy to handle. Just convert to an LDAP filter and do any attribute name mappings required. If you backend data is SQL, it just slightly more difficult to translate the DSML filter into a SQL query clause.
But what if your back end data store doesn’t support a query mechanism? What if the data is in a flat file, or a NOSQL DB? What if the data is only accessible through an API that doesn’t allow for filtering?
There are several ways to solve that problem, but the easiest is to recursively walk the DSML filter and create a decision tree where each node determines if a given instance passes the part of the filter it knows. The code for this is pretty simple in .NET and I posted an example here. Note that this example is just a partial implementation of the SPML search request for the purposes of demonstrating this concept. It is not a full featured implementation of SPML.
The basic idea is that an abstract data provider would return a dictionary of the attribute values for each entry in the data. The interface could look like (in C#):
public interface IUnfilteredDataProvider {
List <DSMLData> DoUnfilteredSearch();
}
In this example the sample data provider reads entries from a flat file. On each search request the filter is recursively read and turned into nodes in a decision tree. Each data entry is then passed to the decision tree and if it passes the filter it is appended to the returned results:
List<DSMLData> dataList = this._unfilteredProvider.DoUnfilteredSearch();
DataFilter df = GetDataFilter(searchRequest);
List<PSOType> returnPSOs = new List<PSOType>();
// return only those entries that pass the filter
foreach (DSMLData data in dataList) {
if (df.Pass(data)) {
returnPSOs.Add(data.GetPSO());
}
}
The GetDataFilter method walks the DSML filter and constructs a decision tree (feel free to download the sample and look at the code for more details). No special meaning given to any of the attributes returned by the provider. They are all just treated as DSML attributes. Of course you will note a potential scalability issue with large data sets, but there are several tricks that can be used to minimize that (thoughts for a later post).
Oh, and this approach works great for creating a DSML service as well and the general concept would be just as easy to implement in Java.
So what does all this mean? Supporting filtered searches in an SPML or DSML service is really not that hard, even if your data is stored in a data store that does not support filtering.
