its length in bytes and lines, its status (contained in the eight
system-defined and eight user-defined flags), and portions of its
RFC-822 header (the "from:", "to:", "date:" and "subject:" fields).
All of this information can be encoded in a small (around 100 bytes)
data structure whose length is independent of the size of the message
it describes.
Most clients should be able to store a complete list of message
descriptors with little problem. This allows a user to get a
complete picture of his mail state without having all his messages
present locally. If a client has extremely limited amounts of disk
storage, it is also possible to get a subset of the descriptors from
the repository. Short messages can reside on the client, along with
the descriptors, and long messages can either be printed via the DMSP
print-message operation, or specially pulled over via the fetch-
message operation.
6. Typical interactive-style client-repository interaction
The following example describes a typical communication session
between the repository and a client mail reader. The client is one
of three belonging to user "Fred". Its name is "office-client", and
since Fred has used the client within the last week, it is marked as
"active". Fred has two mailboxes: "fred" is where all of his
current mail is stored; "archive" is where messages of lasting
importance are kept. The example will run through a simple
synchronization operation. Typically, the synchronization will be
performed by a mail reader as part of a "get new mail" operation.
First Fred's mail reader connects to the repository and receives the
following banner:
200 Pcmail repository version 3.0.0 ready
In order to access his global mail state, the mail reader must
authenticate Fred to the repository; this is done via the DMSP login
operation:
login fred fred-password office-client 0 0
This tells the repository that Fred is logging in via "office-
client", and that "office-client" is identified by an existing client
object in Fred's mail state. The first argument to the login
operation is Fred's repository user name. The second argument is
Fred's password. The third argument is the name of the client
communicating with the repository. The fourth argument tells the
repository not to create "office-client" even if it cannot find its
client object. The final argument tells the repository that Fred's
client is not operating in batch mode but rather in interactive mode.
Fred's authentication checks out, so the repository logs him in.
200 command OK
Now that Fred is logged in, the mail reader performs an initial
synchronization. This process starts with the mail reader's asking
for an up-to-date list of mailboxes:
list-mailboxes
The repository replies with:
230 mailbox list follows:
fred 2313 10 1
archive 101 100 0
.
This tells the mail reader that there are two mailboxes, "fred" and
"archive". "Fred" has 10 messages, one of which is unseen. The next
incoming message will be assigned a UID of 2313. "Archive", on the
other hand, has 100 messages, none of which are unseen. The next
message sent to "archive" will be assigned the UID 101. There are no
new mailboxes in the list (if there were, the mail reader would
create them. On the other hand, if some mailboxes in the mail
reader's local list were not in the repository's list, the program
would assume them deleted by another client and delete them locally
as well).
To synchronize, the mail reader need only look at each mailbox's
contents to see if (1) any new mail has arrived, or (2) if Fred
changed any messages on one of his other two clients subsequent to
"office-client"'s last connection to the repository.
The mail reader asks for any changed descriptors via the "fetch-
changed-descriptors" operation. It requests at most ten changed
descriptors since storage is very limited on Fred's workstation.
fetch-changed-descriptors fred 10
The repository responds with:
250 descriptor list follows:
expunged
2101
expunged
2104
descriptor
2107 1100011100000010 1400 30
foo@bar.edu (Foo Jones)
fred@PTT.LCS.MIT.EDU
Wed, 9 Dec 87 10:43:52 EST
A typical subject line
descriptor
2312 0000000000000000 12232 320
joe@athena.mit.edu
fred@PTT.LCS.MIT.EDU
Thu, 17 Dec 87 18:24:09 PST
Another typical subject line
.
If a descriptor changed because it was expunged, it is transmitted as
two lines: the word "expunged" on one line, followed by the message
UID on the next line. If one of its flags changed state, or it is a
new message, it is transmitted as six lines: the word "descriptor" on
one line, followed by a line containing the message UID, flags, and
length in bytes and lines, followed by the to, from, date, and
subject fields, each on one line. The flags are transmitted as a
single string of ones and zeroes, a one if the flag is on and a zero
if the flag is off. All 16 flags are always transmitted. Flag
zero's state is the first character in the flag string; flag
fifteen's is the last character in the flag string.
The first two descriptors in the list have been expunged, presumably
by Fred's expunging his mailbox on another client. The mail reader
removes messages 2101 and 2104 from its local copy of mailbox "fred".
The next descriptor in the list is one which Fred marked for deletion
on another client yesterday. The mail reader marks the local version
of the message as deleted. The last descriptor in the list is a new
one. The mail reader adds the descriptor to its local list. Since
all changes to mailbox "fred" have now been recorded locally, the
update list can be reset:
reset-descriptors fred 1 2312
The repository responds with:
200 command OK
indicating that it has removed from "office-client"'s update list all
messages in mailbox "fred" with UIDs between 1 and 2312 inclusive (in
this case just two messages). "Fred" has now been synchronized. The
mail reader now turns to Fred's "archive" mailbox and asks for the
first ten changed descriptors.
fetch-changed-descriptors archive 10
The repository responds with:
250 descriptor list follows:
.
The zero-length list tells the mail reader that no descriptors have
been changed in "archive" since its last synchronization. No new
synchronization needs to be performed.
Fred's mail reader is now ready to pull over the new message. The
message is 320 lines long; there might not be sufficient storage on
"office-client" to hold the new message. The mail reader tries
anyway:
fetch-message fred 2312
The repository begins transmitting the message:
251 message follows:
UID: 2312
From: joe@bar.mit.edu
To: fred@PTT.LCS.MIT.EDU
Date: Thu, 17 Dec 87 18:24:09 PST
Subject: Another typical subject line
Fred,
...
Halfway through the message transmission, Fred's workstation runs out
of disk space. Because all DMSP operations are defined to be
failure-atomic, the portion of the message already transmitted is
destroyed locally and the operation fails. The mail reader informs
Fred that the message cannot be pulled over because of a lack of disk
space. The synchronization process is now finished and Fred can
start reading his mail. The new message that was too big to fit on
"office-client" will be marked "off line"; Fred can use the mail
reader to either remote-print it or delete and expunge other messages
until he has enough space to store the new message.
Since Fred is running in interactive mode, changes he makes to any
messages will immediately be transmitted into DMSP operations and
sent to the repository. Depending on the mail reader implementation,
Fred will either have to execute a "synchronize" command periodically
or the client will synchronize for him automatically every so often.
7. A current Pcmail implementation
The following section briefly describes a current Pcmail system that
services a small community of users. The Pcmail repository runs
under UNIX on a DEC Microvax-II connected to the Internet. The
clients run on IBM PCs, XTs, and ATs, as well as Sun workstations,
Microvaxes, and VAX-750s.
7.1. IBM PC client code
Client code for the IBM machines operates only in batch mode. Users
make local state changes in a mail reader; the changes are queued
until the user runs a network client program. The program connects
to the repository, performs the queued changes, and synchronizes
local and global mail states. The network client program then
disconnects from the repository.
The IBM PC client code has gone through several revisions since the
first Pcmail RFCwas published. What was once a fairly primitive and
cumbersome system has evolved into a system that makes excellent use
of the PC's limited resources and provides a fairly powerful, easy-
to-use mail reader.
Users access and modify their local mail state via a mail reader
written in the Epsilon text editor's EEL extension language. Users
are given a variety of commands to operate on individual messages and
mailboxes, as well as to compose outgoing mail.
Synchronization and the processing of queued changes is performed by
a separate program, which the user runs as desired. The program
takes any actions queued while operating the mail reader, and
converts them into DMSP operations. All queued changes are made
before any synchronization is performed. The program can be invoked
directly from the mail reader, without having to exit and restart.
The limitation of IBM PC client operation to batch mode was made
because of development environment limitations. The mail reader
cannot work with the network code inside it because of the network
program architecture. The only solution was to provide a two-part
client, one part of which read the mail and one part of which
interacted with the repository. Although slightly cumbersome, the
two-program setup works quite well.
7.2. UNIX client code
Client code for the Suns, Microvaxes, and VAX-750s runs on 4.2/4.3BSD
UNIX. It is fully interactive, with a powerful mail reader inside
Richard Stallman's GNU-EMACS editor. Since UNIX-based workstations
have a good deal of main memory and disk storage, no effort was made
to lower local mail state size by keeping message descriptors rather
than message text.
The local mail state consists of a number of BABYL-format mailboxes.
The interface is very similar to the RMAIL mail reader already
present in GNU-EMACS.
The mail reader communicates with the repository through network code
implemented in EMACS-LISP. Changes to the local mail state are
immediately made on the repository; although the repository is fast,
there is a small noticeable delay in performing operations over the
network.
There is no provision for automatic synchronization whenever new mail
arrives or old mail is changed by another client. Instead, users
must get any new mail explicitly. A simple "notification" program
runs in the background and wakes up every minute to check for new
mail; when mail arrives, the user executes a command to get the new
mail, synchronizing the mailbox at the same time.
7.3. Repository code
The repository is implemented in C on 4.2/4.3BSD UNIX. Currently it
runs on DEC VAX-750s and Microvaxes, although other repositories will
soon be running on IBM RT machines and Sun workstations. The
repository code is designed to allow several clients belonging to a
particular user to "concurrently" modify the user's state. A locking
scheme prevents one client from modifying mail state while another
client is modifying the same state.
8. Conclusions
Pcmail is now used by a small community of people at the MIT
Laboratory for Computer Science. The repository design works well,
providing an efficient means of storing and maintaining mail state
for several users. Its performance is quite good when up to ten
users are connected; it remains to be seen whether or not the
repository will be efficient at managing the state of ten or a
hundred times that many users. Given sufficient disk storage, it
should be able to, since communication between different users'
clients and the repository is likely to be very asynchronous and
likely to occur in short bursts with long "quiet intervals" in
between as users are busy doing other things.
Members of another research group at LCS are currently working on a
replicated, scalable version of the repository designed to support a
very large community of users with high availability. This
repository also uses DMSP and has successfully communicated with
clients that use the current repository implementation. DMSP
therefore seems to be usable over several flavors of repository
design.
The IBM PC clients are very limited in the way of resources. The
mail reader/editor combination is quite powerful, making local mail
state manipulation fairly easy. Obviously a big performance
enhancement would be to provide a fully interactive client. As it
is, batch-style synchronization is relatively time consuming due to
the low performance of the PCs. The "batch-mode" that the PCs use
tends to be good for those PCs that spend a large percentage of their
time unplugged and away from a network. It is somewhat inconvenient
for those PCs that are always connected to a network and could make
good use of an "interactive-mode" state manipulation.
The UNIX-based clients are more powerful and easier to use than their
PC counterparts. Synchronization is much faster, and there is far
more functionality in the mail reader (having an interface that runs
within GNU-EMACS helps a lot in this respect). Most of those people
using the Pcmail system use the UNIX-based client code.
I. DMSP Protocol Specification
Following are a list of DMSP operations by object type, together with
syntax, and possible responses. Some responses may be followed by
zero or more lines of text, terminated by a single period plus CR-LF
pair. Only success responses and common error responses are listed;
a complete list of possible responses follows this appendix.
Expressions in angle brackets (i.e. <mailbox-name>) are
metalinguistic variables indicating a general request or response
form. Operations with arguments have a sample invocation following
the operation syntax and response.
General operations:
HELP
100 Repository version xxx. Following are supported:
HELP
SEND-VERSION
SEND-MESSAGE
LOGIN
LOGOUT
...
FETCH-MESSAGE
COPY-MESSAGE
.
SEND-VERSION <version-number>
200 Command OK
500 version skew!
i.e. SEND-VERSION 230
SEND-MESSAGE
350 enter message; end with "."
To: markl
From: markl
Subject: a test message
this is a test message
.
Repository responds:
200 Command OK
403 message syntax error
User operations:
LOGIN <user> <password> <client> <create-p> <batch-p>
200 Command OK
221 Client out of date by > 1 week
404 Bad password
405 Client <client-name> is locked
411 No user named <user-name>
421 Client <client-name> not found
i.e. LOGIN markl foo random-client-name 1 0
LOGOUT
200 Command OK
SET-PASSWORD <old-password> <new-password>
200 Command OK
404 Incorrect old password
i.e. SET-PASSWORD foo bar
Client operations:
LIST-CLIENTS
220 Client list <name> <status> follows:
client-1 active
client-2 inactive
client-3 active
...
client-foobar active
.
Each line of the list contains a client name, followed by
whitespace, followed by the word "active" or the word "inactive",
indicating whether or not the client has connected to the repository
within the last week.
CREATE-CLIENT <client-name>
200 Command OK
403 <client-name> is an illegal name
420 Client <client-name> exists
i.e. CREATE-CLIENT new-client
DELETE-CLIENT <client-name>
200 Command OK
421 Client <client-name> not found
405 Client <client-name> is locked
i.e. DELETE-CLIENT old-client
RESET-CLIENT <client-name>
200 Command OK
421 Client <client-name> not found
405 Client <client-name> is locked
i.e. RESET-CLIENT any-old-client
Mailbox operations:
LIST-MAILBOXES
230 Mbox list <name> <high-UID> <#msgs> <#new> follows:
mailbox-1 2338 8 1
mailbox-2 59 44 0
...
mailbox-foobar 19 9 0
.
Each line of the list contains a mailbox name, followed by the
mailbox's next available unique identifier, followed by the number of
messages in the mailbox, followed finally by the number of unseen
messages in the mailbox. Unseen messages are those whose descriptors
have flag #1 ("message has been seen") set to zero.
CREATE-MAILBOX <mailbox-name>
200 Command OK
403 <mailbox-name> is an illegal name
430 <mailbox-name> already exists
440 <mailbox-name> exists as a bboard subscription
i.e. CREATE-MAILBOX current-events
DELETE-MAILBOX <mailbox-name>
200 Command OK
431 mailbox <mailbox-name> not found
440 <mailbox-name> is a bboard; use delete-bboard-mailbox
i.e. DELETE-MAILBOX income-tax-information
CREATE-BBOARD-MAILBOX <mailbox-name>
200 Command OK
430 a mailbox named <mailbox-name> already exists.
430 a bboard mailbox named <mailbox-name> already exists.
403 <mailbox-name> is an illegal name
i.e. CREATE-BBOARD-MAILBOX sf-lovers
DELETE-BBOARD-MAILBOX <mailbox-name>
200 Command OK
404 not owner of <mailbox-name>
431 no bboard mailbox named <mailbox-name>
i.e. DELETE-BBOARD-MAILBOX rec.autos
RESET-MAILBOX <mailbox-name>
200 Command OK
431 mailbox <mailbox-name> not found
i.e. RESET-MAILBOX british-cars
EXPUNGE-MAILBOX <mailbox-name>
200 Command OK
431 mailbox <mailbox-name> not found
EXPUNGE-MAILBOX british-cars
Address operations:
LIST-ADDRESSES <mailbox-name>
260 Address list for <mailbox-name> follows:
address-1
address-2
...
address-6
.
or
431 mailbox <mailbox-name> not found
i.e. LIST-ADDRESSES archive
Each line of the list consists solely of one address.
CREATE-ADDRESS <mailbox-name> <address-name>
200 Command OK
403 <mailbox-name> is an illegal name
431 mailbox <mailbox-name> not found
460 <address-name> already exists
i.e. CREATE-ADDRESS markl markl-bug-pcmail
DELETE-ADDRESS <mailbox-name> <address-name>
200 Command OK
431 mailbox <mailbox-name> not found
461 address <address-name> not found
i.e. DELETE-ADDRESS markl markl-info-cobol
Subscription operations:
LIST-SUBSCRIPTIONS
240 subscription list follows:
bboard-1 2573 33 2606
bboard-2 541 4 545
...
bboard-6 1530 43 1573
.
Each line of the list consists of a bulletin-board name, followed by
the UID of the first message which the user has not yet looked at,
followed by the number of messages in the bulletin-board that the
user has not yet looked at, followed by the bulletin-board's next
available unique message identifier.
CREATE-SUBSCRIPTION <bboard-name>
200 Command OK
403 <bboard-name> is an illegal name
430 A mailbox named <bboard-name> already exists
431 Bboard mailbox <bboard-name> not found
440 Already subscribing to <bboard-name>
i.e. CREATE-SUBSCRIPTION sf-lovers
DELETE-SUBSCRIPTION <bboard-name>
200 Command OK
441 Subscription <bboard-name> not found
i.e. DELETE-SUBSCRIPTION rec.music
RESET-SUBSCRIPTION <bboard-name> <new-UID>
200 Command OK
441 Subscription <bboard-name> not found
i.e. RESET-SUBSCRIPTION rec.music.gdead 1210
LIST-AVAILABLE-SUBSCRIPTIONS
241 All available bboards follow:
mod.politics
sfl
tcp-ip
forum
...
comp.emacs
.
Each line of the list consists solely of one bulletin-board
name.
Message operations:
FETCH-CHANGED-DESCRIPTORS <mailbox-name> <max-to-send>
250 Descriptor list follows:
expunged
2333
expunged
2334
descriptor
2337 0001000001110000 481 14
croaker@ptt.lcs.mit.edu
fred@anymachine.mit.edu
Tue, 19 Jan 88 11:10:03 EST
a typical subject line
descriptor
2339 0000000000000000 1457 40
bob@lcs.mit.edu
csr-people@ptt.lcs.mit.edu
Mon, 18 Jan 88 13:08:17 +0000
another typical subject line
expunged
2340
.
or
431 mailbox <mailbox-name> not found
i.e. FETCH-CHANGED-DESCRIPTORS markl 100
Each element of the descriptor list is either two or six lines long.
Descriptors which have been expunged are transmitted as two lines:
the word "expunged" on one line, followed by the message unique
identifier on the next line. Descriptors which still exist are
transmitted as six lines: the word "descriptor" on one line, followed
by a line containing the message unique identifier, flag states
(sixteen characters either one or zero depending on the associated
flag value), followed by the message length in characters, followed
by the message length in lines. The next four lines contain the
message's "from:", "to:", "date:", and "subject:" fields,
respectively. Flag zero's state is the first character in the flag
string; flag fifteen's is the last character in the flag string.
FETCH-DESCRIPTORS <mailbox-name> <low-uid> <high-uid>
250 Descriptor list follows:
descriptor
2337 0001000001110000 481 14
croaker@ptt.lcs.mit.edu
fred@anymachine.mit.edu
Tue, 19 Jan 88 11:10:03 EST
a typical subject line
descriptor
2339 0000000000000000 1457 40
bob@lcs.mit.edu
csr-people@ptt.lcs.mit.edu
Mon, 18 Jan 88 13:08:17 +0000
another typical subject line
.
or
431 mailbox <mailbox-name> not found
i.e. FETCH-DESCRIPTORS british-cars 12 31
COPY-MESSAGE <src-mailbox> <target-mailbox> <source-UID>
250 Descriptor list follows:
descriptor
2339 0000000000000000 1457 40
bob@lcs.mit.edu
csr-people@ptt.lcs.mit.edu
Mon, 18 Jan 88 13:08:17 +0000
another typical subject line
.
or
400 cannot copy message onto itself
431 target mailbox <target-mailbox> not found
431 source mailbox <source-mailbox> not found
451 message <source-UID> not found
i.e. COPY-MESSAGE markl british-cars 2338
RESET-DESCRIPTORS <mailbox-name> <low-UID> <high-UID>
200 Command OK
431 mailbox <mailbox-name> not found
i.e. RESET-DESCRIPTORS markl 1 10000
PRINT-MESSAGE <mailbox-name> <UID> <printer-ID>
200 Command OK
401 printer <printer-name> not found
431 mailbox <mailbox-name> not found
451 message <UID> not found
i.e. PRINT-MESSAGE markl 2433 pravda
SET-MESSAGE-FLAG <mailbox-name> <UID> <flagnum> <state>
200 Command OK
431 mailbox <mailbox-name> not found
451 message <UID> not found
500 flag number <flag-number> out of range
i.e. SET-MESSAGE-FLAG british-cars 23 0 1
FETCH-MESSAGE <mailbox-name> <UID>
251 message follows:
From: markl@ptt.lcs.mit.edu
To: markl@ptt.lcs.mit.edu
Date: Sun, 17 Jan 88 11:11:11 EST
Subject: anything
this is a sample of some
message text
.
or
431 Mailbox <mailbox-name> not found
451 message <UID> not found
i.e. FETCH-MESSAGE current-events 495
II. Operations by name
copy-message
create-address
create-bboard-mailbox
create-client
create-mailbox
create-subscription
delete-address
delete-bboard-mailbox
delete-client
delete-mailbox
delete-subscription
expunge-mailbox
fetch-changed-descriptors
fetch-descriptors
fetch-message
help
list-addresses
list-available-subscriptions
list-clients
list-mailboxes
list-subscriptions
login
logout
print-message
reset-client
reset-descriptors
reset-mailbox
reset-subscription
send-message
send-version
set-message-flag
set-password
III. Responses by number
100 Pcmail repository version XXX; following are supported
200 Command OK
220 Client list <name> <status> follows:
221 Client out of date by > 1 week
230 Mailbox list <name> <high UID> <#msgs> <#new> follows:
240 Subscription list follows:
250 Descriptor list follows:
251 Message follows:
260 Address list follows:
350 enter message; end with "."
400 cannot copy message onto itself
410 already logged in
420 client <name> already exists
430 mailbox <name> already exists
430 bboard mailbox <name> already exists
440 subscription <name> already exists
460 address <name> already exists
411 no user named <name>
421 client <name> not found
431 mailbox <name> not found
441 subscription <name> not found
451 message <UID> not found
461 address <name> not found
402 internal error message
403 syntax error in outbound message
404 bad password or permission denied
405 mail state is temporarily in use by another client
406 please log in
500 operation syntax error or illegal argument