referrals to DAG-CAPs based on the query types supported by the DAG
itself. The Referral Index creates an index over these objects so
that it can respond to DAG-CAP queries using the DAG/IP. The
information is drawn directly from interactions with participating
WDSPs' software, using the Common Indexing Protocol (CIP).
5.4.2 Interactions with WDSPs (CIP)
WDSPs that wish to participate in the DAG system must register
themselves (see Section 5.4.6). Once registered, the Referral Index
will interact with the WDSPs using the Common Indexing Protocol as
defined in [1], using the Index Object defined in Section 5.4.3.
5.4.3 Index Object Format
The CIP index object type is based on the Tagged Index Object as
defined in [12]. Appendix E details the expected content of the
index objects as they are to be provided by the WDSPs.
TISDAG: The tokens in the Tagged Index Object should be UTF-8
encoded composed UNICODE version 2 character encoding.
5.4.4 DAG-Internal I/O
The Referral Index interacts with the rest of the DAG internal
modules (DAG-CAPs) by listening for queries and responding in the
DAG/IP (defined in Appendix C).
5.4.5 The Index Server
The Referral Index must index the necessary attributes of the CIP
index object in order to respond to queries of the form described in
Table 3.1.
The semantics of the chosen CIP object (defined in Appendix E) are
such that a referral to a WDSP server is sent back if (and only if)
- the index object of the WDSP contains all the tokens of the query,
in the attributes specified, according to the logic of the DAG/IP
query, and
- all of those tokens are found with a common tag.
This means that a query for the name "Fred Flintstone" (2 tokens)
will yield a referral to a server that has a record for "Fred Amadeus
Flintstone", but not to a WDSP with 2 differently tagged records, for
"Fred Amadeus" and "Julie Flintstone". Depending on the access
protocol being used and the original end-user query, the referral to
the WDSP with "Fred Amadeus Flintstone" may yield a successful
result, or it may not. But, it is known that the other WDSP would
not have yielded successful searches. That is, the referral approach
may yield false-positive results, but will not miss appropriate
WDSPs.
5.4.6 Configuration
The Referral Index must provide the ability to register interested
WDSPs, as outlined in Appendix E.
The Referral Index must be able to configure the port for DAG/IP
communications. Also, it must be configurable to recognize only
registered DAG-CAPs.
5.4.7 Security
The Referral Index will accept queries only from recognized
(registered) DAG-CAPs. This will reduce "denial of service" attack
types, but is also a reflection on the fact that the Referral Index
uses the DAG/IP, (i.e., internal) protocol, which should not be
exposed to non-DAG software.
The Referral Index must be able to use authenticated communication to
receive data from WDSPs (see Appendix E).
5.5 Mail (SMTP) DAG-CAP
This is the default Mail DAG-CAP. More sophisticated ones could
certainly be written -- e.g., for pretty-printed output, or for
handling different philosophies of case-matching.
This DAG-CAP has been designed on the assumption that mail queries
will be human-generated (i.e., using a mail program/text editor), as
opposed to being queries formulated by software agents. The input
grammar should therefore be simple and liberal in acceptance of
variations of whitespace formatting.
5.5.1 Mail DAG-CAP Input
Mail DAG-CAP input is expected to be a regular or MIME-encoded (see
[9] and [10]) SMTP mail message, sent to an advertised mail address.
The mail DAG-CAP parses the message and replies to it with a MIME-
encoded message containing the results of the DAG search.
One query is accepted per e-mail message -- text after a single valid
query has been read is simply ignored.
The body of the query message must follow the syntax defined below.
Note that all input control terms ("type=", "name=" etc) are shown in
lower case for convenience, but could be upper case or mixed case on
input.
mailquery = [mnl] [controls] mnl terms mnl
controls = [msp] "searchtype" [msp] "=" [msp]
( matchtype /
casetype /
matchtype msp casetype /
casetype msp matchtype /
<nothing> )
matchtype = "substring" / "exact"
; default: substring
casetype = "ignore" / "sensitive"
; default: ignore
terms = n / n-l / n-o / n-o-l / r-o / r-o-l
n = n-term
n-l = ( n-term l-term / l-term n-term)
n-o = ( n-term o-term / o-term n-term )
n-o-l = ( n-term o-term l-term /
n-term l-term o-term /
l-term n-term o-term /
l-term o-term n-term /
o-term l-term n-term /
o-term n-term l-term )
r-o = ( r-term o-term / o-term r-term )
r-o-l = ( r-term o-term l-term /
r-term l-term o-term /
l-term o-term r-term /
l-term r-term o-term /
o-term l-term r-term /
o-term r-term l-term )
n-term = [msp] "name" [msp] "=" [msp] string mnl
o-term = [msp] "org" [msp] "=" [msp] string mnl
l-term = [msp] "loc" [msp] "=" [msp] string mnl
r-term = [msp] "role" [msp] "=" [msp] string mnl
string = <US-ASCII or quoted-printable encoded
ISO-8859-1 or UTF-8 except nl and sp>
msp = 1*(sp)
sp = " "
mnl = 1*(nl)
nl = <linebreak>
The following are valid mail queries:
Example 1:
searchtype = <NL>
name = thinking cat<NL>
Example 2:
searchtype = exact ignore<NL>
name=thinking cat<NL>
Example 3:
role=thinking cat<NL>
org =space colonization<NL>
Example 4:
name=thinking cat <NL>
<NL>
<NL>
My signature line follows here in the most annoying
fashion <NL>
Note that the following are not acceptable queries:
Example 5:
searchtype= exact substring <NL>
name = thinking cat <NL>
Example 6:
name=thinking cat org= freedom fighters anonymous<NL>
In Example 5, two conflicting searchtypes are given. In Example 6,
no linebreak follows the n-term.
5.5.2 Translation from Mail query to DAG/IP
Querying the Referral Index
A key element of translating from the Mail DAG-CAP input into the
DAG/IP query format is to "tokenize" the input terms into single
token elements for the DAG/IP query. For example, the n-term
name= thinking cat<NL>
is tokenized into 2 n-tokens:
thinking
cat
which are then mapped into the following in the DAG/IP query (dag-n-
terms):
FN=thinking and FN=cat<NL>
The same is true for all r-terms, l-terms and o-terms. The primary
steps in translating the mail input into a DAG/IP query are:
translate quoted-printable encoding, if necessary
translate base64 encoding, if necessary
tokenize the strings for each term
construct the DAG/IP query from the resulting components, as
described in more detail below
DAG/IP constraints are constructed from the searchtype information in
the query.
dag-matchtype = "search=" <matchtype> /
"search=substring" ; if matchtype not
; specified
dag-casetype = "case=ignore" / ; if casetype not
; specified or
; casetype=ignore
"case=consider" ; if casetype=sensitive
constraints = ":" dag-matchtype ";" dag-casetype
The terms for the DAG/IP query are constructed from the tokenized
strings from the mail input.
dag-n-terms = "FN=" n-token 0*( " and FN=" n-token)
dag-o-terms = "ORG=" o-token 0*( " and ORG=" o-token)
dag-l-terms = "LOC=" l-token 0*( " and LOC=" l-token)
dag-r-terms = "ROLE=" r-token 0*( " and ROLE=" r-token)
This means that the relevant DAG/IP queries are formulated as one of
two types:
dagip-query = ( ( ( n-query / nl-query / no-query /
nol-query ) [" and template=DAGPERSON"]":"
dag-matchtype ";" dag-casetype) /
( ( ro-query / rol-query )
[" and template=DAGORGROLE"]":"
dag-matchtype ";" dag-casetype) )
n-query = dag-n-terms
nl-query = dag-n-terms " and " dag-l-terms
no-query = dag-n-terms " and " dag-o-terms
nol-query = dag-n-terms " and " dag-o-terms " and "
dag-l-terms
ro-query = dag-r-terms " and " dag-o-terms
rol-query = dag-r-terms " and " dag-o-terms " and "
dag-l-terms
The examples given earlier are then translated as follows.
Example 1:
FN=thinking and FN=cat:search=substring;case=ignore<NL>
Example 2:
FN=thinking and FN=cat:search=exact;case=ignore<NL>
Example 3:
ROLE=thinking and ROLE=cat and ORG=space and
ORG=colonization:search=substring;case=ignore<NL>
Querying a DAG-SAP
In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
the DAG/IP query must include information about the target WDSP
server. This information is drawn from the Referral Index SERVER-
TO-ASK referral information, and is appended to the query as
specified in Appendix C):
":host=" quoted-hostname ";port=" number ";server-info="
quoted-serverinfo ";charset=" charset
where the response from the Referral Index included:
"# SERVER-TO-ASK " serverhandle nl
" Server-info: " serverinfo nl
" Host-Name: " hostname nl
" Host-Port: " number nl
" Protocol: " prot nl
" Source-URI: " source nl
" Charset: " charset nl
"# END" nl
and the "quoted-hostname" and "quoted-serverinfo" are obtained from
"hostname" and "serverinfo" respectively, by quoting the DAG/IP
special characters.
For example, the referral
# SERVER-TO-ASK dagsystem01<NL>
Server-info: o=thinkingcat, c=se<NL>
Host-Name: thinkingcat.com<NL>
Host-Port: 2839<NL>
Protocol: ldapv2<NL>
Source-URI: http://www.thinkcat.com
Charset: T.61<NL>
# END<NL>
would yield the addition
:host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
c\=se;charset=T\.61
in its query to an LDAPv2 DAG-SAP.
(N.B.: See Appendix C for further definitions of the terms used in
the SERVER-TO-ASK response).
Note that it is the DAG-SAP's responsibility to extract these terms
from the query and use them to identify the WDSP server to be
contacted. See the individual DAG-SAP definitions, below.
5.5.3 Chaining queries in Mail DAG-CAP
The Mail DAG-CAP has to chain all referrals -- to the Whois++ DAG-
SAP, LDAPv2 DAG-SAP, or LDAPv3 DAG-SAP as appropriate for the
referral.
5.5.4 Expression of results in Mail DAG-CAP
The results message is sent to the "Reply-To:" address of the
originating mail, if available (see [4] for appropriate
interpretation of mail originator headers). The original query is
repeated, along with the message-id. The remainder of the body of
the mail message is the concatenation of responses from the DAG-SAP
calls, each result having the WDSP's SOURCE URI (from the referral)
appended to it, and the system messages also having been removed.
At the end of the message, the WDSP servers that failed to respond
(i.e., the DAG-SAP handling the referral returned the "% 403
Information Unavailable" message) are listed with their server-info.
5.5.5 Expression of Errors in Mail DAG-CAP
If the mail DAG-CAP receives a message that is not parsable using the
query grammar described above, it returns an explanatory message to
the query mail's reply address saying that the query could not be
interpreted, and giving a description of valid queries.
If the number of referrals sent by the Referral Index is greater than
the pre-determined maximum (for detecting data-mining efforts, or
otherwise refusing over-general queries, such as "FN=svensson"), the
mail DAG-CAP will send an explanatory message to the query mail's
reply address describing the "over-generalized query" problem,
suggesting the user resubmit a more precise query, and describing the
list of valid query types.
If the mail DAG-CAP receives several different result codes from the
DAG-SAPs it should represent those in an appropriate manner in the
response message.
A mail DAG-CAP may redirect a connection to another mail DAG-CAP for
reasons of load-balancing. This is done simply by forwarding the
mail query to the address of the alternate mail DAG-CAP.
5.6 Web (HTTP) DAG-CAP
5.6.1 Web DAG-CAP Input
The web DAG-CAP provides its interface via standard HTTP protocol.
The general expectation is that the web DAG-CAP will provide a form
page with radio buttons to select "substring or exact match" and
"consider case or ignore case". Other information (about name, role,
organization, locality) is solicited as free-form text.
The DAG-CAP receives queries via an HTTP "post" method (the outcome
of the form action for the page described above, or generated
elsewhere). The rest of this section describes the variables that
are to be expressed in that post. The actual layout of the page and
most user interface issues are left to the discretion of the builder.
Note that the Web DAG-CAP may be called upon to provide responses in
different content encoding, and must therefore address the "Accept-
Encoding:" request header in the HTTP connection.
Although the Web protocol, HTTP, is not itself capable of handling
referrals, through the use of two extra variables this client is
given the option of requesting referral information and then pursuing
individual referrals through the Web DAG-CAP itself, as a proxy for
those referrals. This is handled through the extra "control
variables" to request referrals only, and to indicate when the
transaction is a continuation of a previous query to pursue a
referral.
There has been call to have a "machine-readable" version of the
search output. As HTML is geared towards visual layout, user agents
that intend to do something with the results other than present them
in an HTML browser have few cues to use to extract the relevant
information from the HTML page. Also, "minor" visual changes,
accomplished with extensive HTML updates, can disrupt user agents
that were built to blindly parse the original HTML. Therefore,
provision has been made to return "raw" format results. These are
requested by specifying "Accept-Content: application/whoispp-
response" in the request header of the HTTP message to the HTTP
DAG-CAP.
The variables that are expected are:
transaction = "new" / "chain" ; default is "new". This
; should not be user-settable. It is used
; in constructed URLs
resulttype = "all" / "referrals" ; default is "all"
matchtype = "substring" / "exact"
casetype = "case ignore" / "case sensitive"
n-term = string
o-term = string
l-term = string
r-term = string
host-term = string
port-term = string
servinfo-term = string
prot-term = string ; the protocol of the referral
string = <UNICODE-2-0-UTF-8> / <UNICODE-1-1-UTF-8> /
<ISO-8859-1>
5.6.2 Translation from Web query to DAG/IP
Querying a DAG-SAP Directly
If the transaction variable is "chain", the information in the POST
is used to pursue a particular referral, not do a search of the
Referral Index. The appropriate DAG-SAP (deduced from the prot-term)
is contacted and issued the query directly.
Results from this type of query are always full results (i.e., not
referrals).
Querying the Referral Index
A key element of translating from the Web DAG-CAP input into the
DAG/IP query format is to "tokenize" the input terms into single
token elements for the DAG/IP query. For example, the n-term
name= thinking cat
is tokenized into 2 n-tokens:
thinking
cat
which are then mapped into the following in the DAG/IP query (dag-n-
terms):
FN=thinking and FN=cat
The same is true for the r-term, l-term and o-term.
The primary steps in translating the HTTP input into a DAG/IP query
are:
translate encodings, if necessary
tokenize the strings for each term
construct the DAG/IP query from the resulting components, as
described in more detail below
DAG/IP constraints are constructed from the searchtype information in
the query.
dag-matchtype = "search=" <matchtype> /
"search=substring" ; if matchtype not
; specified
dag-casetype = "case=ignore" / ; if casetype not
; specified or
; casetype="case ignore"
"case=consider" ; if casetype=
; "case sensitive"
constraints = ":" dag-matchtype ";" dag-casetype
The terms for the DAG/IP query are constructed from the tokenized
strings from the HTTP post input.
dag-n-terms = "FN=" n-token 0*( " and FN=" n-token)
dag-o-terms = "ORG=" o-token 0*( " and ORG=" o-token)
dag-l-terms = "LOC=" l-token 0*( " and LOC=" l-token)
dag-r-terms = "ROLE=" r-token 0*( " and ROLE=" r-token)
This means that the relevant DAG/IP queries are formulated as one of
two types:
dagip-query = ( ( ( n-query / nl-query / no-query / nol-query )
[" and template=DAGPERSON"]":" dag-matchtype
";" dag-casetype) /
( ( ro-query / rol-query )
[" and template=DAGORGROLE"]":" dag-matchtype
";" dag-casetype) )
n-query = dag-n-terms
nl-query = dag-n-terms " and " dag-l-terms
no-query = dag-n-terms " and " dag-o-terms
nol-query = dag-n-terms " and " dag-o-terms " and "
dag-l-terms
ro-query = dag-r-terms " and " dag-o-terms
rol-query = dag-r-terms " and " dag-o-terms " and "
dag-l-terms
Querying a DAG-SAP
In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
the DAG/IP query must include information about the target WDSP
server. This information is drawn from the Referral Index SERVER-
TO-ASK referral information, and is appended to the query as
specified in Appendix C:
":host=" quoted-hostname ";port=" number ";server-info="
quoted-serverinfo ";charset=" charset
where the response from the Referral Index included:
"# SERVER-TO-ASK " serverhandle <NL>
" Server-info: " serverinfo <NL>
" Host-Name: " hostname <NL>
" Host-Port: " number <NL>
" Protocol: " prot <NL>
" Source-URI: " source <NL>
" Charset: " charset <NL>
"# END" <NL>
and the "quoted-hostname" and "quoted-serverinfo" are obtained from
"hostname" and "serverinfo" respectively, by quoting the DAG/IP
special characters.
For example, the referral
# SERVER-TO-ASK dagsystem01<NL>
Server-info: o=thinkingcat, c=se<NL>
Host-Name: thinkingcat.com<NL>
Host-Port: 2839<NL>
Protocol: ldapv2<NL>
Source-URI: http://www.thinkingcat.com
Charset: T.61<NL>
# END<NL>
would yield the addition
:host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
c\=se;charset=T\.61
in its query to an LDAPv2 DAG-SAP
(N.B.: See Appendix C for further definitions of the terms used in
the SERVER-TO-ASK response).
Note that it is the DAG-SAP's responsibility to extract these terms
from the query and use them to identify the WDSP server to be
contacted. See the individual DAG-SAP definitions, below.
5.6.3 Chaining queries in Web DAG-CAP
If the resulttype was "all", all of the referrals received from the
Referral Index are chained using the appropriate DAG-SAPs. If only
referrals were requested, the Referral Index results are returned.
5.6.4 Expression of results in Web DAG-CAP
text/html results
The default response encoding is text/html. If the resulttype was
"all", the content of the chaining responses from the DAG-SAPs,
without the system messages, is collated into a single page response,
one result entry per demarcated line ( e.g., bullet item). The FN or
ROLE value should be presented first and clearly. The SOURCE URI for
each WDSP referral should be presented as an HREF for each of the
WDSPs results.
At the end of the message, the WDSP servers that failed to respond
(i.e., the DAG-SAP handling the referral returned the "% 403
Information Unavailable" message) are listed with their server-info.
If, however, the resulttype was "referrals", the results from the
Referral Index are returned as HREF URLs to the Web DAG-CAP itself,
with the necessary information to carry out the query (including the
"HOST=", etc, for the referral).
For example, if the original query:
n-term="thinking cat"
resulttype="referrals"
drew the following referral from the Referral Index:
# SERVER-TO-ASK DAG-Serverhandle<NL>
Server-Info: c=se, o=tce<NL>
Host-Name: answers.tce.com<NL>
Host-Port: 1111<NL>
Protocol: ldapv3<NL>
Source-URI: http://some.service.se/
Charset: UTF-8<NL>
# END<NL>
the response would be an HTML page with an HREF HTTP "POST" URL to
the Web DAG-CAP with the following variables set:
n-term="thinking cat"
transaction="chain"
servinfo-term="c=se, o=tce"
host-term="answers.tce.com"
port-term="1111"
prot-term="ldapv3"
The Source-URI should be established in the response as its own HREF
URI.
application/whoispp-response Results
If Accept-Encoding: " HTTP request header had the value
"application/whoispp-response", the content of the HTTP response will
be constructed in the same syntax and attribute mapping as for the
Whois++ DAG-CAP.
If the resulttype was "all", all the referrals will have been chained
by the Web DAG-CAP, and the response will include only full data
records.
If the resulttype was "referrals", then all referrals are passed
directly back in a single response, in correct Whois++ referral
format (conveniently, this is how they are formulated in the DAG/IP).
Note that this will include referrals to LDAP-based services as well
as Whois++ servers.
5.6.5 Expression of Errors in Web DAG-CAP
A Web DAG-CAP may redirect a connection to another web DAG-CAP for
reasons of load-balancing. This is done simply by using an HTTP
redirect.
Standard Errors
If the web DAG-CAP receives a message that is not parsable using the
query grammar described above, it sends an explanatory HTML page
saying that the query could not be interpreted, and giving a
description of valid queries.
If the number of referrals sent by the Referral Index is greater than
the pre-determined maximum (for detecting data-mining efforts, or
otherwise refusing over-general queries, such as "FN=svensson"), the
web DAG-CAP will send a page with an explanatory message describing
the "over-generalized query" problem, suggesting the user resubmit a
more precise query, and describing the list of valid query types.
If the web DAG-CAP receives more than one result code from the DAG-
SAPs, it must represent them all in a appropriate manner in the
response.
application/whoispp-response Errors
An invalid query is responded to with a simple text response with the
error: "% 500 Syntax Error".
If too many referrals are generated from the Referral Index, the
simple text response will have the message "% 503 Query too general".
5.7 Whois++ DAG-CAP
TISDAG: The system commands polled-for/-by should elicit the empty
set as a return value until we better understand the implications
of doing otherwise.
5.7.1 Whois++ DAG-CAP Input
Input to the Whois++ DAG-CAP follows the Whois++ standard ([6]).
Minimally, the Whois++ DAG-CAP must support the following queries:
Query Type Expression in Whois++
----------- ------------------------------------
N One or more "name=" and
template=USER
NL One or more "name=" and
One or more "address-locality=" and template=USER
NO One or more "name=" and
one or more "organization-name=" and template=USER
NOL One or more "name=" and
one or more "organization-name=" and
one or more "address-locality=" and template=USER
RO One or more "org-role=" and
one or more "organization-name=" and template=ORGROLE
ROL One or more "org-role=" and
one or more "organization-name=" and
one or more "address-locality=" and template=ORGROLE
Table 5.1 Allowable Whois++ Queries
The following constraints must be supported for queries:
"search=" (substring / exact)
"case=" (ignore / consider)
If no constraints are defined in a query the default is exact and
ignore. For example,
FN=foo and loc=kista and fn=bar<NL>
is a perfectly valid Whois++ NL query for "Foo Bar" in "Kista".
5.7.2 Translation from Whois++ query to DAG/IP
Querying the Referral Index
The Whois++ DAG-CAP formulates a DAG/IP query by forwarding the
search terms received (as defined in Table 5.1).
For example, the above query would be expressed as:
FN=foo and LOC=kista and FN=bar and template=DAGPERSON<NL>
Querying a DAG-SAP
In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
the DAG/IP query must include information about the target WDSP
server. This information is drawn from the Referral Index SERVER-
TO-ASK referral information, and is appended to the query as
specified in appendix C:
":host=" quoted-hostname ";port=" number ";server-info="
quoted-serverinfo ";charset=" charset
where the response from the Referral Index included:
"# SERVER-TO-ASK " serverhandle<NL>
" Server-info: " serverinfo<NL>
" Host-Name: " hostname<NL>
" Host-Port: " number<NL>
" Protocol: " prot<NL>
" Source-URI: " source<NL>
" Charset: " charset<NL>
"# END"<NL>
and the "quoted-hostname" and "quoted-serverinfo" are obtained from
"hostname" and "serverinfo" respectively, by quoting the DAG/IP
special characters.
For example, the referral
# SERVER-TO-ASK dagsystem01<NL>
Server-info: o=thinkingcat, c=se<NL>
Host-Name: thinkingcat.com<NL>
Host-Port: 2839<NL>
Protocol: ldapv2<NL>
Source-URI: http://www.thinkingcat.com/
Charset: T.61<NL>
# END<NL>
would yield the addition
:host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
c\=se;charset=T\.61
in its query to an LDAPv2 DAG-SAP.
(N.B.: See Appendix C for further definitions of the terms used in
the SERVER-TO-ASK response).
Note that it is the DAG-SAP's responsibility to extract these terms
from the query and use them to identify the WDSP server to be
contacted. See the individual DAG-SAP definitions, below.
5.7.3 Chaining in Whois++ DAG-CAP
The Whois++ DAG-CAP relies on DAG-SAPs to chain any non-Whois++
referrals (currently, the LDAPv2 and LDAPv3 DAG-SAPs).
5.7.4 Expression of results in Whois++
Results are expressed in Whois++ by collating the DAG/IP results
received from DAG-SAPs (using the FULL response), and using the
template and attribute mappings defined in Appendix B. For each
result from a given referral, the SOURCE attribute is added, with the
value of the SOURCE-URI from the referral.
Any referrals to other Whois++ servers provided by the Referral Index
are sent directly to the Whois++ client as follows:
server-to-ask = "# SERVER-TO-ASK " DAG-Serverhandle<NL>
" Server-Handle: " SERVER-INFO<NL>
" Host-Name: " HOST<NL>
" Host-Port: " PORT<NL>
" Protocol: " PROTOCOL<NL>
"# END"<NL>
where SERVER-INFO, HOST, PORT, PROTOCOL are drawn from the referral
provided in the DAG/IP, and the SOURCE-URI information is lost.
5.7.5 Expression of Errors in Whois++ DAG-CAP
As appropriate, the Whois++ DAG-CAP will express operational errors
following the Whois++ standard. There are 4 particular error
conditions of the DAG system that the DAG-CAP will handle as
described below.
When the Whois++ DAG-CAP receives a query that it cannot reply to
within the (data) constraints of the DAG, it sends an error message
and closes the connection. The error message includes
% 502 Search expression too complicated<NL>
If the number of referrals sent by the Referral Index is greater than
the pre-determined maximum (for detecting data-mining efforts, or
otherwise refusing over-general queries, such as "FN=svensson"), the
Whois++ DAG-CAP will send an error message and close the connection.
The error message includes
% 503 Query too general<NL>
(N.B.: this is different from the "Too many hits" reply, which does
send partial results.)
A Whois++ DAG-CAP may redirect a connection to another Whois++ DAG-
CAP for reasons of load-balancing. This is expressed to the end-user
client software using the SERVER-TO-ASK response with appropriate
information to reach the designated alternate DAG-CAP.
If a Whois++ DAG-CAP receives several different response codes from
DAG-SAPs it should try to represent them all in the response to the
end-user client.
The proposed mapping between DAG/IP response codes and Whois++
response codes are given in Appendix D.
5.8 LDAPv2 DAG-CAP
5.8.1 LDAPv2 DAG-CAP Input
Input to the LDAPv2 DAG-CAP follows the LDAPv2 standard ([19]).
Minimally, the LDAPv2 DAG-CAP must support the following queries
(adapted from the ASN.1 grammar of the standard):
BindRequest ::=
[APPLICATION 0] SEQUENCE {
version INTEGER (1 .. 127),
name LDAPDN,
authentication CHOICE {
simple [0] OCTET STRING,
krbv42LDAP [1] OCTET STRING,
krbv42DSA [2] OCTET STRING
}
}
BindResponse ::= [APPLICATION 1] LDAPResult
SearchRequest ::=
[APPLICATION 3] SEQUENCE {
baseObject "dc=se",
scope wholeSubtree (2),
derefAliases ENUMERATED {
neverDerefAliases (0),
derefInSearching (1),
derefFindingBaseObj (2),
derefAlways (3)
},
sizeLimit INTEGER (0 .. maxInt),
timeLimit INTEGER (0 .. maxInt),
attrsOnly BOOLEAN,
filter Filter,
attributes SEQUENCE OF AttributeType
}
Filter ::=
CHOICE {
and [0] SET OF Filter,
or [1] SET OF Filter,
not [2] Filter,
equalityMatch [3] AttributeValueAssertion,
substrings [4] SubstringFilter
}
SubstringFilter ::=
SEQUENCE {
type AttributeType,
SEQUENCE OF CHOICE {
initial [0] LDAPString,
any [1] LDAPString,
final [2] LDAPString
}
}
Queries against attributes in the prescribed LDAP standard schema
(see Appendix B) are accepted.
N.B., this is a minimal set of supported queries, to achieve the
basic DAG-defined queries. An LDAP DAG-CAP may choose to support
more complex queries than this, if it undertakes to do the
translation from the DAG/IP to the LDAPv2 client in a way that
responds to the semantics of those queries.
TISDAG: Since LDAPv2 didn't specify any characterset but relied
on X.500 to do so, in practice several different charactersets are
in use in Sweden today. That the LDAPv2 CAP has no way of knowing
which characterset that are in use by a connecting client is a
problem that the TISDAG project can not solve.
Users of the DAG system will have to configure their specific
client according to information on the TISDAG web page. That page
provides very specific information (including port number) that
can be given to LDAPv2 users. The LDAP DAG-CAP listening on the
default port (389) will be the LDAPv3 one.
5.8.2 Translation from LDAPv2 query to DAG/IP
Querying the Referral Index
The essential stratagem for mapping LDAP queries into DAG/IP Referral
Index queries is to tokenize the string-oriented LDAP
AttributeValueAssertions or SubstringFilters and construct an
appropriate DAG/IP token-oriented query in the DAG/IP. This will
generalize the LDAP query and yield false-positive referrals, but
should not miss any appropriate referrals.
There are 3 particular cases to be considered:
equalityMatch queries
substring queries
combination equalityMatch and substring queries
TISDAG: If the LDAP filter contains a cn-term and no objectclass
specification it is unclear if the search is for a person or a
role. When this happens the DAG query should cover all bases and
map the query into a query for both people and roles.
EqualityMatch queries can be handled by simply tokenizing the
AttributeValueAssertions, making one DAG/IP query term per token
(using the appropriate DAGSchema attribute) and carrying out an
exact match in the DAG/IP.
Consider the following example, represented in the ASCII
expression of LDAP Filters as described in [13]):
(& (cn=Foo Bar)(objectclass=inetOrgPerson))
This query can be represented in the DAG/IP as
FN="Foo" and FN="Bar":search=exact<NL>
N.B.
The search is set up to be "case=ignore" (the DAG/IP's default)
because the relevant LDAP schema attributes are all derivatives
of the "name" attribute element, which is defined to have a case
insensitive match.
If no objectclass were defined the query in DAG/IP would have
been
(FN="Foo" and FN="bar") or (ROLE="Foo" and ROLE="bar"):search=exact
inetOrgPerson is used as the objectclass in this and the following
examples, although person or organizationalPerson could also have
been used.
This query will yield false-positive referrals; the original
LDAP query should only match against records for which the "cn"
attribute is exactly the phrase "Foo Bar", whereas the DAG/IP
query will yield referrals any WDSP containing records that
include the two tokens "foo" and "bar" in any order.
For example, this DAG/IP query will yield referrals to WDSPs
with records including:
cn: Bar Foo
cn: Le Bar Foo
cn: Foo Bar AB
LDAP substring queries must also be tokenized in order to construct a
DAG/IP query. The additional point to bear in mind is that LDAP
substring expressions are directed at phrases, which obscure
potential token boundaries. Consequently, all points between
substring components must be considered as potential token
boundaries.
Thus, the LDAP query
(& (cn=black) (o=c*t) (objectclass=inetOrgPerson))
could be expressed as a DAG/IP query with 3 tokens, in a substring
search:
FN=black and ORG=c and ORG=t:search=substring<NL>
This query will yield false-positive results as the tokenized query
does not preserve the order of appearance in the LDAP substring, and
it doesn't preserve phrase-boundaries. That is,
ORG=c and ORG=t:search=substring
will match
tabacco
which is not a match by the LDAP query semantics.
Combined EqualityMatch and Substring queries need special attention.
When an LDAP query includes both EqualityMatch components and
substring filter components, the DAG/IP query to the Referral Index
can be constructed by following the same mechanisms of tokenization,
but the whole search will become a substring search, as the DAG/IP
defines only search types across the entire query for Referral Index
queries.
Thus,
(& (cn=Foo Bar) (o=c*t) (objectclass=inetOrgPerson))
can be expressed as
FN=Foo and FN=Bar and ORG=c and ORG=t:search=substring<NL>
Alternatively, the LDAP DAG-CAP could conduct two separate queries
and take the intersection (the logical "AND") of the two sets of
referrals returned by the Referral Index.
Note that DAG/IP can accept phrases for searches -- the query
FN=Foo\ bar<NL> (note the escaped space)
is perfectly valid. However, it would match only those things which
have been tokenized in a way that preserves the space, which is the
empty set in the case of the data stored here.
Querying a DAG-SAP
It is never invalid to use the same substantive query to a DAG-SAP as
was used to obtain referral information from the Referral Index.
However, the over-generalization of these queries may yield excessive
numbers of results, and will necessitate some pruning of results in
order to match the returned results against the semantics of the
original LDAP query. It is the LDAP DAG-CAP that is responsible for
this pruning, as it is the recipient of the original query, and
responsible for responding to its semantics.
In concrete terms, when making the DAG/IP query which is to be sent
to a DAG-SAP the above mentioned queries are still valid queries,
but an alternative finer-grained query is also possible, namely:
FN=foo and FN=bar and ORG=c;search=lstring and ORG=t;search=tstring
Particularly in the case of the LDAPv2 DAG-CAP, however, there will
be cause to use LDAP(v2/v3) DAG-SAPs. Since these DAG-SAPs also deal
in phrase-oriented data, a less-over-generalized query can be passed
to them:
FN=Foo\ Bar:search=exact<NL>
In querying a DAG-SAP (irrespective of the protocol of that DAG-SAP),
the DAG/IP query must include information about the target WDSP
server. This information is drawn from the Referral Index SERVER-
TO-ASK referral information, and is appended to the query as
specified in Appendix C:
":host=" quoted-hostname ";port=" number ";server-info="
quoted-serverinfo ";charset=" charset
where the response from the Referral Index included:
"# SERVER-TO-ASK " serverhandle<NL>
" Server-info: " serverinfo<NL>
" Host-Name: " hostname<NL>
" Host-Port: " number<NL>
" Protocol: " prot<NL>
" Source-URI: " source<NL>
" Charset: " charset<NL>
"# END<NL>
and the "quoted-hostname" and "quoted-serverinfo" are obtained from
"hostname" and "serverinfo" respectively, by quoting the DAG/IP
special characters.
For example, the referral
# SERVER-TO-ASK dagsystem01<NL>
Server-info: o=thinkingcat, c=se<NL>
Host-Name: thinkingcat.com<NL>
Host-Port: 2839<NL>
Protocol: ldapv2<NL>
Source-URI: http://www.thinkingcat.com <NL>
Charset: T.61<NL>
# END<NL>
would yield the addition
:host=thinkingcat\.com;port=2839;server-info=o\=thinkingcat\,\
c\=se;charset=T\.61
in its query to an LDAPv2 DAG-SAP.
(N.B.: See Appendix C for further definitions of the terms used in
the SERVER-TO-ASK response).
Note that it is the DAG-SAP's responsibility to extract these terms
from the query and use them to identify the WDSP server to be
contacted. See the individual DAG-SAP definitions, below.
5.8.3 Chaining queries in LDAPv2 DAG-CAP
The LDAPv2 DAG-CAP relies on DAG-SAPs to resolve every referral.
5.8.4 Expression of results in LDAPv2
As described above, results from DAG-SAPs will have to be post-
processed in cases where the original query was generalized for
expression in DAG/IP.
Acceptable results are expressed in the LDAP search response:
SearchResponse ::=
CHOICE {
entry [APPLICATION 4] SEQUENCE {
objectName LDAPDN,
attributes SEQUENCE OF SEQUENCE
{
AttributeType,
SET OF AttributeValue
}
},
resultCode [APPLICATION 5] LDAPResult
}
where
LDAPDN = DN / "cn=" (FN/ROLE) [",o="ORG] ",dc=se"
attributes = <all attributes mapped from DAG schema, and
"objectClass = inetOrgPerson",
"objectClass = top",
"objectClass = person" or
"objectClass = organizationalRole", as
appropriate, and "labeledURI = <SOURCE-URI>"
for each result from a given referral>
(Where DN,FN,ORG and ROLE are the values from the DAG schema).
I.e., where available, the entry's true DN is used; otherwise (e.g.,
for data coming from Whois++ servers), a reasonable facsimile is
constructed.
5.8.5 Expression of Errors in LDAPv2 DAG-CAP
As appropriate, the LDAPv2 DAG-CAP will express system responses
following the LDAPv2 standard.
Appendix D gives the proposed mapping between DAG/IP response codes
and LDAPv2 resultcodes.
There are 4 particular error conditions of the DAG system that the
DAG-CAP will handle as described below.
When the LDAPv2 DAG-CAP receives a query that it cannot reply to
within the (data) constraints of the DAG queries, it sends an error
message and closes the connection. The error message includes the
LDAPv2 resultCode:
noSuchAttribute (for incorrect schema attributes)
inappropriateMatching (when a match type other than those
supported is used, e.g. approxMatch)
unwillingToPerform (when the query is not one of the
defined types)
If the number of referrals sent by the Referral Index is greater than
the pre-determined maximum (for detecting data-mining efforts, or
otherwise refusing over-general queries, such as "FN=svensson"), the
LDAPv2 DAG-CAP will send an error message. The error message
includes one of the following resultCodes:
sizeLimitExceeded
timeLimitExceeded
An LDAPv2 DAG-CAP may redirect a connection to another LDAPv2 DAG-CAP
for reasons of load-balancing. This is expressed to the end-user
client software using the "umich referral" convention to direct the
client software to an alternate DAG-CAP by passing the URL in an
error message.
Since a LDAPv2 DAG-CAP only can send one resultcode back to a client;
If a LDAPv2 DAG-CAP receives several different result codes from the
DAG-SAPs it will have to construct a resultmessage that to some
extent represents the combination of those. It is proposed that in
these cases the following actions are taken:
- All the response codes are collected
- Each response code are translated into the corresponding LDAPv2
resultcode.
- A resultcode is chosen to represent the collected response on the
following grounds:
If "success" is the only resultcode represented after these
steps the return that result code.
If apart from "success" there is one other resultcode represented
return that other resultcode.
If apart from "success" there are two or more resultcodes
represented return the resultcode "other".
5.9 LDAPv3 DAG-CAP
5.9.1 LDAPv3 DAG-CAP Input
Input to the LDAPv3 DAG-CAP follows the LDAPv3 definition (currently
defined in [17]). Minimally, the LDAPv3 DAG-CAP must support the
following queries (adapted from the ASN.1 grammar of the standard):
BindRequest ::= [APPLICATION 0] SEQUENCE {
version INTEGER (1 .. 127),
name LDAPDN,
authentication AuthenticationChoice }
AuthenticationChoice ::= CHOICE {
simple [0] OCTET STRING,
-- 1 and 2 reserved
sasl [3] SaslCredentials }
SaslCredentials ::= SEQUENCE {
mechanism LDAPString,
credentials OCTET STRING OPTIONAL }
BindResponse ::= [APPLICATION 1] SEQUENCE {
COMPONENTS OF LDAPResult,
serverSaslCreds [7] OCTET STRING OPTIONAL }
SearchRequest ::= [APPLICATION 3] SEQUENCE {
baseObject c=se,
scope wholeSubtree (2) },
derefAliases ENUMERATED {
neverDerefAliases (0),
derefInSearching (1),
derefFindingBaseObj (2),
derefAlways (3) },
sizeLimit INTEGER (0 .. maxInt),
timeLimit INTEGER (0 .. maxInt),
typesOnly BOOLEAN,
filter Filter,
attributes AttributeDescriptionList }
Filter ::= CHOICE {
and [0] SET OF Filter,
or [1] SET OF Filter,
not [2] Filter,
equalityMatch [3] AttributeValueAssertion,
substrings [4] SubstringFilter }
SubstringFilter ::= SEQUENCE {
type AttributeDescription,
-- at least one must be present
substrings initial [0] LDAPString,
substrings any [1] LDAPString,
substrings final [2] LDAPString}
Queries against attributes in the proscribed LDAP standard schema
(see Appendix B) are accepted.
N.B., this is a minimal set of supported queries, to achieve the
basic DAG-defined queries. An LDAP DAG-CAP may choose to support
more complex queries than this, if it undertakes to do the
translation from the DAG/IP to the LDAPv3 client in a way that
responds to the semantics of those queries.
5.9.2 Translation from LDAPv3 query to DAG/IP
Querying the Referral Index
The essential stratagem for mapping LDAP queries into DAG/IP Referral
Index queries is to tokenize the string-oriented LDAP
AttributeValueAssertions or SubstringFilters and construct an
appropriate DAG/IP token-oriented query in the DAGschema. This will
generalize the LDAP query and yield false-positive referrals, but
should not miss any appropriate referrals.
There are 3 particular cases to be considered:
equalityMatch queries
substring queries
combination equalityMatch and substring queries
TISDAG: If the LDAP filter contains a cn-term and no objectclass
specification it is unclear if the search is for a person or a
role. When this happens the DAG query should cover all bases and
map the query into a query for both people and roles.
EqualityMatch queries can be handled by simply tokenizing the
AttributeValueAssertions, making one DAG/IP query term per token
(using the appropriate DAGSchema attribute) and carrying out an exact
match in the DAG/IP.
Consider the following example, represented in the ASCII expression
of LDAP Filters as described in [13]):
(& (cn=Foo Bar)(objectclass=person))
This query can be represented in the DAG/IP as
FN="Foo" and FN="Bar":search=exact<NL>
N.B.
The search is set up to be "case=ignore" (the DAG/IP's default)
because the relevant LDAP schema attributes are all derivatives of
the "name" attribute element, which is defined to have a case
insensitive match.
If no objectclass where defined the query in DAG/IP would have been
(FN="Foo" and FN="bar") or ( ROLE="Foo" and ROLE="bar"):search=exact
Although person is used as objectclass in this and the following
examples, inetOrgPerson or organizationalPerson could also have been
used.
This query will yield false-positive referrals; the original LDAP
query should only match against records for which the "cn" attribute
is exactly the phrase "Foo Bar", whereas the DAG/IP query will yield
referrals any WDSP containing records that include the two tokens
"foo" and "bar" in any order.
For example, this DAG/IP query will yield referrals to WDSPs with
records including:
cn: Bar Foo
cn: Le Bar Foo
cn: Foo Bar AB
LDAP substring queries must also be tokenized in order to construct a
DAG/IP query. The additional point to bear in mind is that LDAP
substring expressions are directed at phrases, which obscure
potential token boundaries. Consequently, all points between
substring components must be considered as potential token
boundaries.
Thus, the LDAP query
(& (cn=black) o=c*t) (objectclass=person))
should be expressed as a DAG/IP query with 3 tokens, in a substring
search:
FN=black and ORG=c and ORG=t:search=substring<NL>
This query will yield false-positive results as the tokenized query
does not preserve the order of appearance in the LDAP substring, and
it doesn't preserve phrase-boundaries. That is,
ORG=c and ORG=t:search=substring
will match
tabacco
which is not a match by the LDAP query semantics.
Combined EqualityMatch and Substring queries need special attention.
When an LDAP query includes both EqualityMatch components and
substring filter components, the DAG/IP query to the Referral Index
can be constructed by following the same mechanisms of tokenization,
but the whole search will become a substring search, as the DAG/IP
defines search types across the entire query.
Thus,
(& (cn=Foo Bar) (o=c*t) (objectclass=person))
can be expressed as
FN=Foo and FN=Bar and ORG=c and ORG=t:search=substring<NL>
Alternatively, the LDAP DAG-CAP could conduct two separate queries
and take the intersection (the logical "AND") of the two sets of
referrals returned by the Referral Index.
Note that DAG/IP can accept phrases for searches -- the query
FN=Foo\ bar<NL> (note the escaped space)
is perfectly valid. However, it would match only those things which
have been tokenized in a way that preserves the space, which is the
empty set in the case of the data stored here.
Querying a DAG-SAP
It is never invalid to use the same substantive query to a DAG-SAP as
was used to obtain referral information from the Referral Index.