extensions, the entire upload is accomplished in a single
command/response round-trip.
Note: Client implementers should be aware that [MULTIAPPEND] performs
append of multiple messages atomically. This means, for example, if
there is not enough space to save "n"-th message (or the message has
invalid structure and is rejected by the server) after successful
upload of "n-1" messages, the whole upload operation fails, and no
message will be saved in the mailbox. Although this behavior might
be desirable in certain situations, it might not be what you want.
Otherwise, the client should use the regular APPEND command (Section
4.2.2.3), possibly utilizing the [LITERAL+] extension. See also
Section 5.1 for discussions about error recovery.
Note: MULTIAPPEND can be used together with the UIDPLUS extension in
a way similar to what was described in Section 4.2.1. [MULTIAPPEND]
extends the syntax of the APPENDUID response code to allow for
multiple message UIDs in the second parameter.
Example 2:
This example demonstrates the use of MULTIAPPEND together with
UIDPLUS (synchronization points where the client waits for
confirmations from the server are marked with "<--->"):
C: A003 APPEND Jan-2002 (\Seen $MDNSent) "31-May-2002 05:26:59 -0600"
{310}
<--->
S: + Ready for literal data
C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)
C: From: Fred Foobar <foobar@blt.example.COM>
C: Subject: afternoon meeting
C: To: mooch@owatagu.siam.edu
C: Message-Id: <B27397-0100000@blt.example.COM>
C: MIME-Version: 1.0
C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
C:
C: Hello Joe, do you think we can meet at 3:30 tomorrow?
C: (\Seen) " 1-Jun-2002 22:43:04 -0800" {286}
<--->
S: + Ready for literal data
C: Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST)
C: From: Joe Mooch <mooch@OWaTaGu.siam.EDU>
C: Subject: Re: afternoon meeting
C: To: foobar@blt.example.com
C: Message-Id: <a0434793874930@OWaTaGu.siam.EDU>
C: MIME-Version: 1.0
C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
C:
C: 3:30 is fine with me.
C:
S: A003 OK [APPENDUID 1022843275 77712,77713] completed
The upload takes 3 round-trips.
Example 3:
In this example, Example 2 was modified for the case when the server
supports MULTIAPPEND, LITERAL+, and UIDPLUS. The upload takes only 1
round-trip.
C: A003 APPEND Jan-2002 (\Seen $MDNSent) "31-May-2002 05:26:59 -0600"
{310+}
C: Date: Mon, 7 Feb 1994 21:52:25 -0800 (PST)
C: From: Fred Foobar <foobar@blt.example.COM>
C: Subject: afternoon meeting
C: To: mooch@owatagu.siam.edu
C: Message-Id: <B27397-0100000@blt.example.COM>
C: MIME-Version: 1.0
C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
C:
C: Hello Joe, do you think we can meet at 3:30 tomorrow?
C: (\Seen) " 1-Jun-2002 22:43:04 -0800" {286+}
C: Date: Mon, 7 Feb 1994 22:43:04 -0800 (PST)
C: From: Joe Mooch <mooch@OWaTaGu.siam.EDU>
C: Subject: Re: afternoon meeting
C: To: foobar@blt.example.com
C: Message-Id: <a0434793874930@OWaTaGu.siam.EDU>
C: MIME-Version: 1.0
C: Content-Type: TEXT/PLAIN; CHARSET=US-ASCII
C:
C: 3:30 is fine with me.
C:
S: A003 OK [APPENDUID 1022843275 77712,77713] completed
4.2.3. Replaying Local Flag Changes
The disconnected client uses the STORE command to synchronize local
flag state with the server. The disconnected client SHOULD use
+FLAGS.SILENT or -FLAGS.SILENT in order to set or unset flags
modified by the user while offline. The FLAGS form MUST NOT be used,
as there is a risk that this will overwrite flags on the server that
have been changed by some other client.
Example 4:
For the message with UID 15, the disconnected client stores the
following flags \Seen and $Highest. The flags were modified on the
server by some other client: \Seen, \Answered, and $Highest. While
offline, the user requested that the $Highest flags be removed and
that the \Deleted flag be added. The flag synchronization sequence
for the message should look like:
C: A001 UID STORE 15 +FLAGS.SILENT (\Deleted)
S: A001 STORE completed
C: A002 UID STORE 15 -FLAGS.SILENT ($Highest)
S: A002 STORE completed
If the disconnected client is able to store an additional binary
state information (or a piece of information that can take a value
from a predefined set of values) in the local cache of an IMAP
mailbox or in a local mailbox (e.g., message priority), and if the
server supports storing of arbitrary keywords, the client MUST use
keywords to store this state on the server.
Example 5:
Imagine a speculative mail client that can mark a message as one of
work-related ($Work), personal ($Personal), or spam ($Spam). In
order to mark a message as personal, the client issues:
C: A001 UID STORE 15 +FLAGS.SILENT ($Personal)
S: A001 STORE completed
C: A002 UID STORE 15 -FLAGS.SILENT ($Work $Spam)
S: A002 STORE completed
In order to mark the message as not work, not personal and not spam,
the client issues:
C: A003 UID STORE 15 -FLAGS.SILENT ($Personal $Work $Spam)
S: A003 STORE completed
4.2.4. Processing Mailbox Compression (EXPUNGE) Requests
A naive disconnected client implementation that supports compressing
a mailbox while offline may decide to issue an EXPUNGE command to the
server in order to expunge messages marked \Deleted. The problem
with this command during synchronization is that it permanently
erases all messages with the \Deleted flag set, i.e., even those
messages that were marked as \Deleted on the server while the user
was offline. Doing this might result in an unpleasant surprise for
the user.
Fortunately the [UIDPLUS] extension can help in this case as well.
The extension introduces UID EXPUNGE command, that, unlike EXPUNGE,
takes a UID set parameter, that lists UIDs of all messages that can
be expunged. When processing this command the server erases only
messages with \Deleted flag listed in the UID list. Thus, messages
not listed in the UID set will not be expunged even if they have the
\Deleted flag set.
Example 6:
While the user was offline, 3 messages with UIDs 7, 27, and 65 were
marked \Deleted when the user requested to compress the open mailbox.
Another client marked a message \Deleted on the server (UID 34).
During synchronization, the disconnected client issues:
C: A001 UID EXPUNGE 7,27,65
S: * ... EXPUNGE
S: * ... EXPUNGE
S: * ... EXPUNGE
S: A001 UID EXPUNGE completed
If another client issues UID SEARCH DELETED command (to find all
messages with the \Deleted flag) before and after the UID EXPUNGE, it
will get:
Before:
C: B001 UID SEARCH DELETED
S: * SEARCH 65 34 27 7
S: B001 UID SEARCH completed
After:
C: B002 UID SEARCH DELETED
S: * SEARCH 34
S: B002 UID SEARCH completed
In the absence of the [UIDPLUS] extension, the following sequence of
commands can be used as an approximation. Note: It’s possible for
another client to mark additional messages as deleted while this
sequence is being performed. In this case, these additional messages
will be expunged as well.
1) Find all messages marked \Deleted on the server.
C: A001 UID SEARCH DELETED
S: * SEARCH 65 34 27 7
S: A001 UID SEARCH completed
2) Find all messages that must not be erased (for the previous
example the list will consist of the message with UID 34).
3) Temporarily remove \Deleted flag on all messages found in step 2.
C: A002 UID STORE 34 -FLAGS.SILENT (\Deleted)
S: A002 UID STORE completed
4) Expunge the mailbox.
C: A003 EXPUNGE
S: * 20 EXPUNGE
S: * 7 EXPUNGE
S: * 1 EXPUNGE
S: A003 EXPUNGE completed
Here, the message with UID 7 has message number 1, with UID 27 has
message number 7, and with UID 65 has message number 20.
5) Restore \Deleted flag on all messages found when performing step
2.
C: A004 UID STORE 34 +FLAGS.SILENT (\Deleted)
S: A004 UID STORE completed
4.2.5. Closing a Mailbox
When the disconnected client has to close a mailbox, it should not
use the CLOSE command, because CLOSE does a silent EXPUNGE. (Section
4.2.4 explains why EXPUNGE should not be used by a disconnected
client.) It is safe to use CLOSE only if the mailbox was opened with
EXAMINE.
If the mailbox was opened with SELECT, the client can use one of the
following commands to implicitly close the mailbox and prevent the
silent expunge:
1) UNSELECT - This is a command described in [UNSELECT] that works as
CLOSE, but doesn’t cause the silent EXPUNGE. This command is
supported by the server if it reports UNSELECT in its CAPABILITY
list.
2) SELECT <another_mailbox> - SELECT causes implicit CLOSE without
EXPUNGE.
3) If the client intends to issue LOGOUT after closing the mailbox,
it may just issue LOGOUT, because LOGOUT causes implicit CLOSE
without EXPUNGE as well.
4) SELECT <non_existing_mailbox> - If the client knows a mailbox that
doesn’t exist or can’t be selected, it MAY SELECT it.
If the client opened the mailbox with SELECT and just wants to avoid
implicit EXPUNGE without closing the mailbox, it may also use the
following:
5) EXAMINE <mailbox> - Reselect the same mailbox in read-only mode.
4.3. Details of "Normal" Synchronization of a Single Mailbox
The most common form of synchronization is where the human trusts the
integrity of the client’s copy of the state of a particular mailbox
and simply wants to bring the client’s cache up to date so that it
accurately reflects the mailbox’s current state on the server.
4.3.1. Discovering New Messages and Changes to Old Messages
Let <lastseenuid> represent the highest UID that the client knows
about in this mailbox. Since UIDs are allocated in strictly
ascending order, this is simply the UID of the last message in the
mailbox that the client knows about. Let <lastseenuid+1> represent
<lastseenuid>’s UID plus one. Let <descriptors> represent a list
consisting of all the FETCH data item items that the implementation
considers part of the descriptor; at a minimum this is just the FLAGS
data item, but it usually also includes BODYSTRUCTURE and
RFC822.SIZE. At this step, <descriptors> SHOULD NOT include RFC822.
With no further information, the client can issue the following two
commands:
tag1 UID FETCH <lastseenuid+1>:* <descriptors>
tag2 UID FETCH 1:<lastseenuid> FLAGS
The first command will request some information about "new" messages
(i.e., messages received by the server since the last
synchronization). It will also allow the client to build a message
number to UID map (only for new messages). The second command allows
the client to
1) update cached flags for old messages;
2) find out which old messages got expunged; and
3) build a mapping between message numbers and UIDs (for old
messages).
The order here is significant. We want the server to start returning
the list of new message descriptors as fast as it can, so that the
client can start issuing more FETCH commands, so we start out by
asking for the descriptors of all the messages we know the client
cannot possibly have cached yet. The second command fetches the
information we need to determine what changes may have occurred to
messages that the client already has cached. Note that the former
command should only be issued if the UIDNEXT value cached by the
client differs from the one returned by the server. Once the client
has issued these two commands, there’s nothing more the client can do
with this mailbox until the responses to the first command start
arriving. A clever synchronization program might use this time to
fetch its local cache state from disk or to start the process of
synchronizing another mailbox.
The following is an example of the first FETCH:
C: A011 UID fetch 131:* (FLAGS BODYSTRUCTURE INTERNALDATE
RFC822.SIZE)
Note 1: The first FETCH may result in the server’s sending a huge
volume of data. A smart disconnected client should use message
ranges (see also Section 3.2.1.2 of [RFC2683]), so that the user is
able to execute a different operation between fetching information
for a group of new messages.
Example 7:
Knowing the new UIDNEXT returned by the server on SELECT or EXAMINE
(<uidnext>), the client can split the UID range
<lastseenuid+1>:<uidnext> into groups, e.g., 100 messages. After
that, the client can issue:
C: A011 UID fetch <lastseenuid+1>:<lastseenuid+100>
(FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE)
...
C: A012 UID fetch <lastseenuid+101>:<lastseenuid+200>
(FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE)
...
...
C: A0FF UID fetch <lastseenuid+901>:<uidnext>
(FLAGS BODYSTRUCTURE INTERNALDATE RFC822.SIZE)
Note that unless a SEARCH command is issued, it is impossible to
determine how many messages will fall into a subrange, as UIDs are
not necessarily contiguous.
Note 2: The client SHOULD ignore any unsolicited EXPUNGE responses
received during the first FETCH command. EXPUNGE responses contain
message numbers that are useless to a client that doesn’t have the
message-number-to-UID translation table.
The second FETCH command will result in zero or more untagged fetch
responses. Each response will have a corresponding UID FETCH data
item. All messages that didn’t have a matching untagged FETCH
response MUST be removed from the local cache.
For example, if the <lastseenuid> had a value 15000 and the local
cache contained 3 messages with the UIDs 12, 777, and 14999,
respectively, then after receiving the following responses from the
server, the client must remove the message with UID 14999 from its
local cache.
S: * 1 FETCH (UID 12 FLAGS (\Seen))
S: * 2 FETCH (UID 777 FLAGS (\Answered \Deleted))
Note 3: If the client is not interested in flag changes (i.e., the
client only wants to know which old messages are still on the
server), the second FETCH command can be substituted with:
tag2 UID SEARCH UID 1:<lastseenuid>
This command will generate less traffic. However, an implementor
should be aware that in order to build the mapping table from message
numbers to UIDs, the output of the SEARCH command MUST be sorted
first, because there is no requirement for a server to return UIDs in
SEARCH response in any particular order.
4.3.2. Searching for "Interesting" Messages.
This step is performed entirely on the client (from the information
received in the step described in 4.3.1), entirely on the server, or
on some combination of both. The decision on what is an
"interesting" message is up to the client software and the human.
One easy criterion that should probably be implemented in any client
is whether the message is "too big" for automatic retrieval, where
"too big" is a parameter defined in the client’s configuration.
Another commonly used criterion is the age of a message. For
example, the client may choose to download only messages received in
the last week (in this case, <date> would be today’s date minus 7
days):
tag3 UID SEARCH UID <uidset> SINCE <date>
Keep in mind that a date search disregards time and time zone. The
client can avoid doing this search if it specified INTERNALDATE in
<descriptors> on the step described in 4.3.1. If the client did, it
can perform the local search on its message cache.
At this step, the client also decides what kind of information about
a particular message to fetch from the server. In particular, even
for a message that is considered "too big", the client MAY choose to
fetch some part(s) of it. For example, if the message is a
multipart/mixed containing a text part and a MPEG attachment, there
is no reason for the client not to fetch the text part. The decision
of which part should or should not be fetched can be based on the
information received in the BODYSTRUCTURE FETCH response data item
(i.e., if BODYSTRUCTURE was included in <descriptors> on the step
described in 4.3.1).
4.3.3. Populating Cache with "Interesting" Messages.
Once the client has found out which messages are "interesting", it
can start issuing appropriate FETCH commands for "interesting"
messages or parts thereof.
Note that fetching a message into the disconnected client’s local
cache does NOT imply that the human has (or even will) read the
message. Thus, the synchronization program for a disconnected client
should always be careful to use the .PEEK variants of the FETCH data
items that implicitly set the \Seen flag.
Once the last descriptor has arrived and the last FETCH command has
been issued, the client simply needs to process the incoming fetch
items and use them to update the local message cache.
In order to avoid deadlock problems, the client must give processing
of received messages priority over issuing new FETCH commands during
this synchronization process. This may necessitate temporary local
queuing of FETCH requests that cannot be issued without causing a
deadlock. In order to achieve the best use of the "expensive"
network connection, the client will almost certainly need to pay
careful attention to any flow-control information that it can obtain
from the underlying transport connection (usually a TCP connection).
Note: The requirement stated in the previous paragraph might result
in an unpleasant user experience, if followed blindly. For example,
the user might be unwilling to wait for the client to finish
synchronization before starting to process the user’s requests. A
smart disconnected client should allow the user to perform requested
operations in between IMAP commands that are part of the
synchronization process. See also Note 1 in Section 4.3.1.
Example 8:
After fetching a message BODYSTRUCTURE, the client discovers a
complex MIME message. Then, it decides to fetch MIME headers of the
nested MIME messages and some body parts.
C: A011 UID fetch 11 (BODYSTRUCTURE)
S: ...
C: A012 UID fetch 11 (BODY[HEADER] BODY[1.MIME] BODY[1.1.MIME]
BODY[1.2.MIME] BODY[2.MIME] BODY[3.MIME] BODY[4.MIME]
BODY[5.MIME] BODY[6.MIME] BODY[7.MIME] BODY[8.MIME] BODY[9.MIME]
BODY[10.MIME] BODY[11.MIME] BODY[12.MIME] BODY[13.MIME]
BODY[14.MIME] BODY[15.MIME] BODY[16.MIME] BODY[17.MIME]
BODY[18.MIME] BODY[19.MIME] BODY[20.MIME] BODY[21.MIME])
S: ...
C: A013 UID fetch 11 (BODY[1.1] BODY[1.2])
S: ...
C: A014 UID fetch 11 (BODY[3] BODY[4] BODY[5] BODY[6] BODY[7] BODY[8]
BODY[9] BODY[10] BODY[11] BODY[13] BODY[14] BODY[15] BODY[16]
BODY[21])
S: ...
4.3.4. User-Initiated Synchronization
After the client has finished the main synchronization process as
described in Sections 4.3.1-4.3.3, the user may optionally request
additional synchronization steps while the client is still online.
This is not any different from the process described in Sections
4.3.2 and 4.3.3.
Typical examples are:
1) fetch all messages selected in UI.
2) fetch all messages marked as \Flagged on the server.
4.4. Special Case: Descriptor-Only Synchronization
For some mailboxes, fetching the descriptors might be the entire
synchronization step. Practical experience with IMAP has shown that
a certain class of mailboxes (e.g., "archival" mailboxes) are used
primarily for long-term storage of important messages that the human
wants to have instantly available on demand but does not want
cluttering up the disconnected client’s cache at any other time.
Messages in this kind of mailbox would be fetched exclusively by
explicit actions queued by the local MUA. Thus, the only
synchronization desirable on this kind of mailbox is fetching enough
descriptor information for the user to be able to identify messages
for subsequent download.
Special mailboxes that receive messages from a high volume, low
priority mailing list might also be in this category, at least when
the human is in a hurry.
4.5. Special Case: Fast New-Only Synchronization
In some cases, the human might be in such a hurry that he or she
doesn’t care about changes to old messages, just about new messages.
In this case, the client can skip the UID FETCH command that obtains
the flags and UIDs for old messages (1:<lastseenuid>).
4.6. Special Case: Blind FETCH
In some cases, the human may know (for whatever reason) that he or
she always wants to fetch any new messages in a particular mailbox,
unconditionally. In this case, the client can just fetch the
messages themselves, rather than just the descriptors, by using a
command like:
tag1 UID FETCH <lastseenuid+1>:* (FLAGS BODY.PEEK[])
Note that this example ignores the fact that the messages can be
arbitrary long. The disconnected client MUST always check for
message size before downloading, unless explicitly told otherwise. A
well-behaved client should instead use something like the following:
1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS RFC822.SIZE)".
2) From the message sizes returned in step 1, construct UID set
<required_messages>.
3) Issue "tag2 UID FETCH <required_messages> (BODY.PEEK[])".
or
1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS)".
2) Construct UID set <old_uids> from the responses of step 1.
3) Issue "tag2 SEARCH UID <old_uids> SMALLER <message_limit>".
Construct UID set <required_messages> from the result of the
SEARCH command.
4) Issue "tag3 UID FETCH <required_messages> (BODY.PEEK[])".
or
1) Issue "tag1 UID FETCH <lastseenuid+1>:* (FLAGS
BODY.PEEK[]<0.<length>>)", where <length> should be replaced with
the maximal message size the client is willing to download.
Note: In response to such a command, the server will only return
partial data if the message is longer than <length>. It will
return the full message data for any message whose size is smaller
than or equal to <length>. In the former case, the client will
not be able to extract the full MIME structure of the message from
the truncated data, so the client should include BODYSTRUCTURE in
the UID FETCH command as well.
5. Implementation Considerations
Below are listed some common implementation pitfalls that should be
considered when implementing a disconnected client.
1) Implementing fake UIDs on the client.
A message scheduled to be uploaded has no UID, as UIDs are
selected by the server. The client may implement fake UIDs
internally in order to reference not-yet-uploaded messages in
further operations. (For example, a message could be scheduled to
be uploaded, but subsequently marked as deleted or copied to
another mailbox). Here, the client MUST NOT under any
circumstances send these fake UIDs to the server. Also, client
implementers should be reminded that according to [IMAP4] a UID is
a 32-bit unsigned integer excluding 0. So, both 4294967295 and
2147483648 are valid UIDs, and 0 and -1 are both invalid. Some
disconnected mail clients have been known to send negative numbers
(e.g., "-1") as message UIDs to servers during synchronization.
Situation 1: The user starts composing a new message, edits it,
saves it, continues to edit it, and saves it again.
A disconnected client may record in its replay log (log of
operations to be replayed on the server during synchronization)
the sequence of operations as shown below. For the purpose of
this situation, we assume that all draft messages are stored in