Request for Comments: 4533 OpenLDAP Foundation
Category: Experimental J.H. Choi
IBM Corporation
June 2006
The Lightweight Directory Access Protocol (LDAP)
Content Synchronization Operation
Status of This Memo
This memo defines an Experimental Protocol for the Internet
community. It does not specify an Internet standard of any kind.
Discussion and suggestions for improvement are requested.
Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2006).
IESG Note
The IESG notes that this work was originally discussed in the LDUP
working group. The group came to consensus on a different approach,
documented in RFC 3928; that document is on the standards track and
should be reviewed by those considering implementation of this
proposal.
Abstract
This specification describes the Lightweight Directory Access
Protocol (LDAP) Content Synchronization Operation. The operation
allows a client to maintain a copy of a fragment of the Directory
Information Tree (DIT). It supports both polling for changes and
listening for changes. The operation is defined as an extension of
the LDAP Search Operation.
Table of Contents
1. Introduction ....................................................3
1.1. Background .................................................3
1.2. Intended Usage .............................................4
1.3. Overview ...................................................5
1.4. Conventions ................................................8
2. Elements of the Sync Operation ..................................8
2.1. Common ASN.1 Elements ......................................9
2.2. Sync Request Control .......................................9
2.3. Sync State Control ........................................10
2.4. Sync Done Control .........................................10
2.5. Sync Info Message .........................................11
2.6. Sync Result Codes .........................................11
3. Content Synchronization ........................................11
3.1. Synchronization Session ...................................12
3.2. Content Determination .....................................12
3.3. refreshOnly Mode ..........................................13
3.4. refreshAndPersist Mode ....................................16
3.5. Search Request Parameters .................................17
3.6. objectName ................................................18
3.7. Canceling the Sync Operation ..............................19
3.8. Refresh Required ..........................................19
3.9. Chattiness Considerations .................................20
3.10. Operation Multiplexing ...................................21
4. Meta Information Considerations ................................22
4.1. Entry DN ..................................................22
4.2. Operational Attributes ....................................22
4.3. Collective Attributes .....................................23
4.4. Access and Other Administrative Controls ..................23
5. Interaction with Other Controls ................................23
5.1. ManageDsaIT Control .......................................24
5.2. Subentries Control ........................................24
6. Shadowing Considerations .......................................24
7. Security Considerations ........................................25
8. IANA Considerations ............................................26
8.1. Object Identifier .........................................26
8.2. LDAP Protocol Mechanism ...................................26
8.3. LDAP Result Codes .........................................26
9. Acknowledgements ...............................................26
10. Normative References ..........................................27
11. Informative References ........................................28
Appendix A. CSN-based Implementation Considerations ..............29
1. Introduction
The Lightweight Directory Access Protocol (LDAP) [RFC4510] provides a
mechanism, the search operation [RFC4511], that allows a client to
request directory content matching a complex set of assertions and to
request that the server return this content, subject to access
control and other restrictions, to the client. However, LDAP does
not provide (despite the introduction of numerous extensions in this
area) an effective and efficient mechanism for maintaining
synchronized copies of directory content. This document introduces a
new mechanism specifically designed to meet the content
synchronization requirements of sophisticated directory applications.
This document defines the LDAP Content Synchronization Operation, or
Sync Operation for short, which allows a client to maintain a
synchronized copy of a fragment of a Directory Information Tree
(DIT). The Sync Operation is defined as a set of controls and other
protocol elements that extend the Search Operation.
1.1. Background
Over the years, a number of content synchronization approaches have
been suggested for use in LDAP directory services. These approaches
are inadequate for one or more of the following reasons:
- failure to ensure a reasonable level of convergence;
- failure to detect that convergence cannot be achieved (without
reload);
- require pre-arranged synchronization agreements;
- require the server to maintain histories of past changes to DIT
content and/or meta information;
- require the server to maintain synchronization state on a per-
client basis; and/or
- are overly chatty.
The Sync Operation provides eventual convergence of synchronized
content when possible and, when not, notification that a full reload
is required.
The Sync Operation does not require pre-arranged synchronization
agreements.
The Sync Operation does not require that servers maintain or use any
history of past changes to the DIT or to meta information. However,
servers may maintain and use histories (e.g., change logs,
tombstones, DIT snapshots) to reduce the number of messages generated
and to reduce their size. As it is not always feasible to maintain
and use histories, the operation may be implemented using purely
(current) state-based approaches. The Sync Operation allows use of
either the state-based approach or the history-based approach on an
operation-by-operation basis to balance the size of history and the
amount of traffic. The Sync Operation also allows the combined use
of the state-based and the history-based approaches.
The Sync Operation does not require that servers maintain
synchronization state on a per-client basis. However, servers may
maintain and use per-client state information to reduce the number of
messages generated and the size of such messages.
A synchronization mechanism can be considered overly chatty when
synchronization traffic is not reasonably bounded. The Sync
Operation traffic is bounded by the size of updated (or new) entries
and the number of unchanged entries in the content. The operation is
designed to avoid full content exchanges, even when the history
information available to the server is insufficient to determine the
client’s state. The operation is also designed to avoid transmission
of out-of-content history information, as its size is not bounded by
the content and it is not always feasible to transmit such history
information due to security reasons.
This document includes a number of non-normative appendices providing
additional information to server implementors.
1.2. Intended Usage
The Sync Operation is intended to be used in applications requiring
eventually-convergent content synchronization. Upon completion of
each synchronization stage of the operation, all information to
construct a synchronized client copy of the content has been provided
to the client or the client has been notified that a complete content
reload is necessary. Except for transient inconsistencies due to
concurrent operation (or other) processing at the server, the client
copy is an accurate reflection of the content held by the server.
Transient inconsistencies will be resolved by subsequent
synchronization operations.
Possible uses include the following:
- White page service applications may use the Sync Operation to
maintain a current copy of a DIT fragment, for example, a mail
user agent that uses the sync operation to maintain a local
copy of an enterprise address book.
- Meta-information engines may use the Sync Operation to maintain
a copy of a DIT fragment.
- Caching proxy services may use the Sync Operation to maintain a
coherent content cache.
- Lightweight master-slave replication between heterogeneous
directory servers. For example, the Sync Operation can be used
by a slave server to maintain a shadow copy of a DIT fragment.
(Note: The International Telephone Union (ITU) has defined the
X.500 Directory [X.500] Information Shadowing Protocol (DISP)
[X.525], which may be used for master-slave replication between
directory servers. Other experimental LDAP replication
protocols also exist.)
This protocol is not intended to be used in applications requiring
transactional data consistency.
As this protocol transfers all visible values of entries belonging to
the content upon change instead of change deltas, this protocol is
not appropriate for bandwidth-challenged applications or deployments.
1.3. Overview
This section provides an overview of basic ways the Sync Operation
can be used to maintain a synchronized client copy of a DIT fragment.
- Polling for changes: refreshOnly mode
- Listening for changes: refreshAndPersist mode
1.3.1. Polling for Changes (refreshOnly)
To obtain its initial client copy, the client issues a Sync request:
a search request with the Sync Request Control with mode set to
refreshOnly. The server, much like it would with a normal search
operation, returns (subject to access controls and other
restrictions) the content matching the search criteria (baseObject,
scope, filter, attributes). Additionally, with each entry returned,
the server provides a Sync State Control indicating state add. This
control contains the Universally Unique Identifier (UUID) [UUID] of
the entry [RFC4530]. Unlike the Distinguished Name (DN), which may
change over time, an entry’s UUID is stable. The initial content is
followed by a SearchResultDone with a Sync Done Control. The Sync
Done Control provides a syncCookie. The syncCookie represents
session state.
To poll for updates to the client copy, the client reissues the Sync
Operation with the syncCookie previously returned. The server, much
as it would with a normal search operation, determines which content
would be returned as if the operation were a normal search operation.
However, using the syncCookie as an indicator of what content the
client was sent previously, the server sends copies of entries that
have changed with a Sync State Control indicating state add. For
each changed entry, all (modified or unmodified) attributes belonging
to the content are sent.
The server may perform either or both of the two distinct
synchronization phases that are distinguished by how to synchronize
entries deleted from the content: the present and the delete phases.
When the server uses a single phase for the refresh stage, each phase
is marked as ended by a SearchResultDone with a Sync Done Control. A
present phase is identified by a FALSE refreshDeletes value in the
Sync Done Control. A delete phase is identified by a TRUE
refreshDeletes value. The present phase may be followed by a delete
phase. The two phases are delimited by a refreshPresent Sync Info
Message having a FALSE refreshDone value. In the case that both the
phases are used, the present phase is used to bring the client copy
up to the state at which the subsequent delete phase can begin.
In the present phase, the server sends an empty entry (i.e., no
attributes) with a Sync State Control indicating state present for
each unchanged entry.
The delete phase may be used when the server can reliably determine
which entries in the prior client copy are no longer present in the
content and the number of such entries is less than or equal to the
number of unchanged entries. In the delete mode, the server sends an
empty entry with a Sync State Control indicating state delete for
each entry that is no longer in the content, instead of returning an
empty entry with state present for each present entry.
The server may send syncIdSet Sync Info Messages containing the set
of UUIDs of either unchanged present entries or deleted entries,
instead of sending multiple individual messages. If refreshDeletes
of syncIdSet is set to FALSE, the UUIDs of unchanged present entries
are contained in the syncUUIDs set; if refreshDeletes of syncIdSet is
set to TRUE, the UUIDs of the entries no longer present in the
content are contained in the syncUUIDs set. An optional cookie can
be included in the syncIdSet to represent the state of the content
after synchronizing the presence or the absence of the entries
contained in the syncUUIDs set.
The synchronized copy of the DIT fragment is constructed by the
client.
If refreshDeletes of syncDoneValue is FALSE, the new copy includes
all changed entries returned by the reissued Sync Operation, as well
as all unchanged entries identified as being present by the reissued
Sync Operation, but whose content is provided by the previous Sync
Operation. The unchanged entries not identified as being present are
deleted from the client content. They had been either deleted,
moved, or otherwise scoped-out from the content.
If refreshDeletes of syncDoneValue is TRUE, the new copy includes all
changed entries returned by the reissued Sync Operation, as well as
all other entries of the previous copy except for those that are
identified as having been deleted from the content.
The client can, at some later time, re-poll for changes to this
synchronized client copy.
1.3.2. Listening for Changes (refreshAndPersist)
Polling for changes can be expensive in terms of server, client, and
network resources. The refreshAndPersist mode allows for active
updates of changed entries in the content.
By selecting the refreshAndPersist mode, the client requests that the
server send updates of entries that are changed after the initial
refresh content is determined. Instead of sending a SearchResultDone
Message as in polling, the server sends a Sync Info Message to the
client indicating that the refresh stage is complete and then enters
the persist stage. After receipt of this Sync Info Message, the
client will construct a synchronized copy as described in Section
1.3.1.
The server may then send change notifications as the result of the
original Sync search request, which now remains persistent in the
server. For entries to be added to the returned content, the server
sends a SearchResultEntry (with attributes) with a Sync State Control
indicating state add. For entries to be deleted from the content,
the server sends a SearchResultEntry containing no attributes and a
Sync State Control indicating state delete. For entries to be
modified in the return content, the server sends a SearchResultEntry
(with attributes) with a Sync State Control indicating state modify.
Upon modification of an entry, all (modified or unmodified)
attributes belonging to the content are sent.
Note that renaming an entry of the DIT may cause an add state change
where the entry is renamed into the content, a delete state change
where the entry is renamed out of the content, and a modify state
change where the entry remains in the content. Also note that a
modification of an entry of the DIT may cause an add, delete, or
modify state change to the content.
Upon receipt of a change notification, the client updates its copy of
the content.
If the server desires to update the syncCookie during the persist
stage, it may include the syncCookie in any Sync State Control or
Sync Info Message returned.
The operation persists until canceled [RFC3909] by the client or
terminated by the server. A Sync Done Control shall be attached to
SearchResultDone Message to provide a new syncCookie.
1.4. Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14 [RFC2119].
Protocol elements are described using ASN.1 [X.680] with implicit
tags. The term "BER-encoded" means the element is to be encoded
using the Basic Encoding Rules [X.690] under the restrictions
detailed in Section 5.1 of [RFC4511].
2. Elements of the Sync Operation
The Sync Operation is defined as an extension to the LDAP Search
Operation [RFC4511] where the directory user agent (DUA or client)
submits a SearchRequest Message with a Sync Request Control and the
directory system agent (DSA or server) responds with zero or more
SearchResultEntry Messages, each with a Sync State Control; zero or
more SearchResultReference Messages, each with a Sync State Control;
zero or more Sync Info Intermediate Response Messages; and a
SearchResultDone Message with a Sync Done Control.
To allow clients to discover support for this operation, servers
implementing this operation SHOULD publish 1.3.6.1.4.1.4203.1.9.1.1
as a value of the ’supportedControl’ attribute [RFC4512] of the root
DSA-specific entry (DSE). A server MAY choose to advertise this
extension only when the client is authorized to use it.
2.1. Common ASN.1 Elements
2.1.1. syncUUID
The syncUUID data type is an OCTET STRING holding a 128-bit
(16-octet) Universally Unique Identifier (UUID) [UUID].
syncUUID ::= OCTET STRING (SIZE(16))
-- constrained to UUID
2.1.2. syncCookie
The syncCookie is a notational convenience to indicate that, while
the syncCookie type is encoded as an OCTET STRING, its value is an
opaque value containing information about the synchronization session
and its state. Generally, the session information would include a
hash of the operation parameters that the server requires not be
changed and the synchronization state information would include a
commit (log) sequence number, a change sequence number, or a time
stamp. For convenience of description, the term "no cookie" refers
either to a null cookie or to a cookie with pre-initialized
synchronization state.
syncCookie ::= OCTET STRING
2.2. Sync Request Control
The Sync Request Control is an LDAP Control [RFC4511] where the
controlType is the object identifier 1.3.6.1.4.1.4203.1.9.1.1 and the
controlValue, an OCTET STRING, contains a BER-encoded
syncRequestValue. The criticality field is either TRUE or FALSE.
syncRequestValue ::= SEQUENCE {
mode ENUMERATED {
-- 0 unused
refreshOnly (1),
-- 2 reserved
refreshAndPersist (3)
},
cookie syncCookie OPTIONAL,
reloadHint BOOLEAN DEFAULT FALSE
}
The Sync Request Control is only applicable to the SearchRequest
Message.
2.3. Sync State Control
The Sync State Control is an LDAP Control [RFC4511] where the
controlType is the object identifier 1.3.6.1.4.1.4203.1.9.1.2 and the
controlValue, an OCTET STRING, contains a BER-encoded syncStateValue.
The criticality is FALSE.
syncStateValue ::= SEQUENCE {
state ENUMERATED {
present (0),
add (1),
modify (2),
delete (3)
},
entryUUID syncUUID,
cookie syncCookie OPTIONAL
}
The Sync State Control is only applicable to SearchResultEntry and
SearchResultReference Messages.
2.4. Sync Done Control
The Sync Done Control is an LDAP Control [RFC4511] where the
controlType is the object identifier 1.3.6.1.4.1.4203.1.9.1.3 and the
controlValue contains a BER-encoded syncDoneValue. The criticality
is FALSE (and hence absent).
syncDoneValue ::= SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDeletes BOOLEAN DEFAULT FALSE
}
The Sync Done Control is only applicable to the SearchResultDone
Message.
2.5. Sync Info Message
The Sync Info Message is an LDAP Intermediate Response Message
[RFC4511] where responseName is the object identifier
1.3.6.1.4.1.4203.1.9.1.4 and responseValue contains a BER-encoded
syncInfoValue. The criticality is FALSE (and hence absent).
syncInfoValue ::= CHOICE {
newcookie [0] syncCookie,
refreshDelete [1] SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDone BOOLEAN DEFAULT TRUE
},
refreshPresent [2] SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDone BOOLEAN DEFAULT TRUE
},
syncIdSet [3] SEQUENCE {
cookie syncCookie OPTIONAL,
refreshDeletes BOOLEAN DEFAULT FALSE,
syncUUIDs SET OF syncUUID
}
}
2.6. Sync Result Codes
The following LDAP resultCode [RFC4511] is defined:
e-syncRefreshRequired (4096)
3. Content Synchronization
The Sync Operation is invoked when the client sends a SearchRequest
Message with a Sync Request Control.
The absence of a cookie or an initialized synchronization state in a
cookie indicates a request for initial content, while the presence of
a cookie representing a state of a client copy indicates a request
for a content update. Synchronization Sessions are discussed in
Section 3.1. Content Determination is discussed in Section 3.2.
The mode is either refreshOnly or refreshAndPersist. The refreshOnly
and refreshAndPersist modes are discussed in Sections 3.3 and 3.4,
respectively. The refreshOnly mode consists only of a refresh stage,
while the refreshAndPersist mode consists of a refresh stage and a
subsequent persist stage.
3.1. Synchronization Session
A sequence of Sync Operations where the last cookie returned by the
server for one operation is provided by the client in the next
operation is said to belong to the same Synchronization Session.
The client MUST specify the same content-controlling parameters (see
Section 3.5) in each Search Request of the session. The client
SHOULD also issue each Sync request of a session under the same
authentication and authorization associations with equivalent
integrity and protections. If the server does not recognize the
request cookie or the request is made under different associations or
non-equivalent protections, the server SHALL return the initial
content as if no cookie had been provided or return an empty content
with the e-syncRefreshRequired LDAP result code. The decision
between the return of the initial content and the return of the empty
content with the e-syncRefreshRequired result code MAY be based on
reloadHint in the Sync Request Control from the client. If the
server recognizes the request cookie as representing empty or initial